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1 . 1 Intended Audience 

These set of lecture notes are designed for an upper-division undergraduate 
course on computational physics. 

1.2 Major Sources 

The sources which I have consulted most frequently whilst developing course 
material are as follows: 

C/C+ + programming: 

Software engineering in C, P.A. Darnell, and P.E. Margolis (Springer-Verlag, 
New York NY, 1988). 

The C++ programming language, 2nd edition, B. Stroustrup (Addison-Wesley 
Reading MA, 1991). 

Schaum's outline: Programming with C, 2nd edition, B. Gottfried (McGraw- 
Hill, New York NY, 1996). 

Schaum's outline: Programming with C++, 2nd edition, J.R. Hubbard (McGraw- 
Hill, New York NY, 2000). 

Numerical methods and computational physics: 

Computational physics, D. Potter (Wiley, New York NY, 1973). 

Numerical recipes in C: the art of scientific computing, W.H. Press, S.A. Teukol- 
sky W.T. Vettering, and B.R. Flannery (Cambridge University Press, Cam- 
bridge UK, 1992). 

Computational physics, N.J. Giordano (Prentice-Hall, Upper Saddle River NJ, 
1997). 

Numerical methods for physics, 2nd edition, A.L. Garcia (Prentice-Hall, Upper 
Saddle River NJ, 2000). 
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Physics of baseball: 

The physics of baseball, R.K. Adair (Harper & Row, New York NY, 1990). 

The physics of sports, A.A. Armenti, Jr., Ed. (American Institute of Physics, 
New York NY, 1992). 

Chaos: 

Chaos in a computer-animated pendulum, R.L. Kautz, Am. J. Phys. 61, 407 
(1993). 

Nonlinear dynamics and chaos, S.H. Strogatz, (Addison-Wesley Reading MA, 
1994). 

Chaos: An introduction to dynamical systems, K.T. Alligood, T.D. Sauer, and 
J. A. Yorke, (Springer-Verlag, New York NY, 1997). 

1.3 Purpose of Course 

The purpose of this course is demonstrate to students how computers can enable 
us to both broaden and deepen our understanding of physics by vastly increasing 
the range of mathematical calculations which we can conveniently perform. 

1.4 Course Philosophy 

My approach to computational physics is to write self-contained programs in a 
high-level scientific language — i.e., either FORTRAN or C/C+ + . Of course, there 
are many other possible approaches, each with their own peculiar advantages 
and disadvantages. It is instructive to briefly examine the available options. 

1.5 Programming Methodologies 

Basically, there are three possible methods by which we could perform the nu- 
merical calculations which we are going to encouter during this course. 



9 



1.5 Programming Methodologies 



1 INTRODUCTION 



firstly, we could use a mathematical software package, such as Mathemat- 
ical], Maple@ or Matlab@ The main advantage of these packages is that they 
facilitate the very rapid coding up of numerical problems. The main disadvan- 
tage is that they produce executable code which is interpreted, rather than com- 
piled. Compiled code is translated directly from a high-level language into ma- 
chine code instructions, which, by definition, are platform dependent — after all, 
an Intel x86 chip has a completely different instruction set to a Power-PC chip. 
Interpreted code is translated from a high-level language into a set of meta-code 
instructions which are platform independent. Each meta-code instruction is then 
translated into a fixed set of machine code instructions which is peculiar to the 
particular hardware platform on which the code is being run. In general, inter- 
preted code is nowhere near as efficient, in terms of computer resource utiliza- 
tion, as compiled code: i.e., interpreted code run a lot slower than equivalent 
compiled code. Thus, although Mathematica, Maple, and Matlab are ideal 
environments in which to perform relatively small calculations, they are not suit- 
able for full-blown research projects, since the code which they produce generally 
runs far too slowly. 

Secondly, we could write our own programs in a high-level language, but use 
calls to pre-written, pre-compiled routines in commonly available subroutine li- 
braries, such as NAGE UNPACK^ and ODEPACK0 to perform all of the real 
numerical work. This is the approach used by the majority of research physicists. 

Thirdly, we could write our own programs — completely from scratch — in a 
high-level language. This is the approach used in this course. I have opted not to 
use pre-written subroutine libraries, simply because I want students to develop 
the ability to think for themselves about scientific programming and numerical 
techniques. Students should, however, realize that, in many cases, pre-written 
library routines offer solutions to numerical problems which are pretty hard to 
improve upon. 



1 See http : //www. wolf ram. com 
2 See http : //www.maplesof t . com 
3 See http://www.mathworks.com 
4 See http://www.nag.com 
5 See http://www.netlib.org 
6 ibid. 
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1.6 Scientific Programming Languages 

What is the best high-level language to use for scientific programming? This, 
unfortunately, is a highly contentious question. Over the years, literally hundreds 
of high-level languages have been developed. However, few have stood the test of 
time. Many languages (e.g., Algol, Pascal, Haskell) can be dismissed as ephemeral 
computer science fads. Others (e.g., Cobol, Lisp, Ada) are too specialized to adapt 
for scientific use. Let us examine the remaining options: 

FORTRAN 77: FORTRAN was the first high-level programming language to be 
developed: in fact, it predates the languages listed below by decades. Be- 
fore the advent of FORTRAN, all programming was done in assembler code! 
Moreover, FORTRAN was specifically designed for scientific computing. In- 
deed, in the early days of computers all computing was scientific in nature — 
i.e., physicists and mathematicians were the original computer scientists! 
FORTRAN'S main advantages are that it is very straightforward, and it in- 
terfaces well with most commonly available, pre-written subroutine libraries 
(since these libraries generally consist of compiled FORTRAN code) . FOR- 
TRAN'S main disadvantages are all associated with its relative antiquity. For 
instance. FORTRAN'S control statements are fairly rudimentary, whereas its 
input/output facilities are positively paleolithic. 

FORTRAN 90: This language is a major extension to FORTRAN 77 which does 
away with many of the latter language's objectionable features. In addition, 
many "modern" features, such as dynamic memory allocation, are included 
in the language for the first time. The major disadvantage of this language 
is the absence of an inexpensive compiler. There seems little prospect of this 
situation changing in the near future. 

C: This language was originally developed by computer scientists to write op- 
erating systems. Indeed, all UNIX operating systems are written in C. C is, 
consequently, an extremely flexible and powerful language. Amongst its ma- 
jor advantages are its good control statements and excellent input/output 
facilities. C's main disadvantage is that, since it was not specifically written 
to be a scientific language, some important scientific features (e.g., complex 
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arithmetic) are missing. Although C is a high-level language, it incorporates 
many comparatively low-level features, such as pointers (this is hardly sur- 
prisingly, since C was originally designed to write operating systems) . The 
low-level features of C — in particular, the rather primitive implementation 
of arrays — sometimes make scientific programming more complicated than 
need be the case, and undoubtedly facilitate programming errors. On the 
other hand, these features allow scientific programmers to write extremely 
efficient code. Since efficiency is generally the most important concern in 
scientific computing, the low-level features of C are, on balance, advanta- 
geous. 

C + + : This language is a major extension of C whose main aim is to facilitate 
object-orientated programming. Object-orientation is a completely different 
approach to programming than the more traditional procedural approach: 
it is particularly well suited to large projects involving many people who 
are each writing different segments of the same code. However, object- 
orientation represents a large, and somewhat unnecessary, overhead for the 
type of straightforward, single person programming tasks considered in this 
course. Note, however, that C+ + incorporates some non-object-orientated 
extensions to C which are extremely useful. 

Of the above languages, we can immediately rule out C++, because object- 
orientation is an unnecessary complication (at least, for our purposes), and FOR- 
TRAN 90, because of the absence of an inexpensive compiler. The remaining 
options are FORTRAN 77 and C. I have chosen to use C (augmented by some of 
the useful, non-object-orientated features of C + + ) in this course, simply because 
I find the archaic features of FORTRAN 77 too embarrassing to teach students in 
the 21st century. 
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2 Scientific Programming in C 

2.1 Introduction 

As we have already mentioned, C is a flexible, extremely powerful, high-level 
programming language which was initially designed for writing operating sys- 
tems and system applications. In fact, all UNIX operating systems, as well as 
most UNIX applications (e.g., text editors, window managers, etc.) are written 
in C. However, C is also an excellent vehicle for scientific programming, since, 
almost by definition, a good scientific programming language must be powerful, 
flexible, and high-level. Having said this, many of the features of C which send 
computer scientists into raptures are not particularly relevant to the needs of the 
scientific programmer. Hence, in the following, we shall only describe that subset 
of the C language which is really necessary to write scientific programs. It may be 
objected that our cut-down version of C bears a suspicious resemblance to FOR- 
TRAN. However, this resemblance is hardly surprising. After all, FORTRAN is a 
high-level programming language which was specifically designed with scientific 
computing in mind. 

As discussed previously, C + + is an extension of the C language whose main 
aim is to facilitate object-orientated programming. The object-orientated features 
of C + + are superfluous to our needs in this course. However, C+ + incorporates 
some new, non-object-orientated features which are extremely useful to the sci- 
entific programmer. We shall briefly discuss these features towards the end of this 
section. Finally, we shall describe some prewritten C+ + classes which allow us 
to incorporate complex arithmetic (which is not part of the C language), variable 
size arrays, and graphics into our programs. 

2.2 Variables 

Variable names in C can consist of letters and numbers in any order, except that 
the first character must be a letter. Names are case sensitive, so upper- and lower- 
case letters are not interchangeable. The underscore character (_) can also be 
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included in variable names, and is treated as a letter. There is no restriction on 
the length of names in C. Of course, variable names are not allowed to clash with 
keywords that play a special role in the C language, such as int, double, if, 
return, void, etc. The following are examples of valid variable names in C: 

x cl4 area electron_mass TEMPERATURE 

The C language supports a great variety of different data types. However, the 
two data types which occur most often in scientific programs are integer, denoted 
int, and floating-point, denoted double. (Note that variables of the most basic 
floating-point data type float are not generally stored to sufficient precision by 
the computer to be of much use in scientific programming.) The data type (int 
or double) of every variable in a C program must be declared before that variable 
can appear in an executable statement. 

Integer constants in C are denoted, in the regular fashion, by strings of arabic 
numbers: e.g., 

57 4567 128933 

Floating-point constants can be written in either regular or scientific notation: e.g., 

0.01 70.456 3e+5 .5067e-16 

Strings are mainly used in scientific programs for data input and output pur- 
poses. A string consists of any number of consecutive characters (including 
blanks) enclosed in double quotation marks: e.g., 

"red" "Austin TX, 78723" "512-926-1477" 

Line-feeds can be incorporated into strings via the escape sequence \n: e.g., 

"Line l\nLine 2\nLine 3" 

The above string would be displayed on a computer terminal as 

Line 1 
Line 2 
Line 3 
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A declaration associates a group of variables with a specific data type. As 
mentioned previously, all variables must be declared before they can appear in 
executable statements. A declaration consists of a data type followed by one or 
more variable names, ending in a semicolon. For instance, 



In the above, a, b, and c are declared to be integer variables, whereas acc, 
epsilon, and t are declared to be floating-point variables. 

A type declaration can also be used to assign initial values to variables. Some 
examples of how to do this are given below: 

int a = 3, b = 5; 
double factor = 1.2E-5; 

Here, the integer variables a and b are assigned the initial values 3 and 5, re- 
spectively, whereas the floating-point variable factor is assigned the initial value 

1.2 x 1CT 5 . 

Note that there is no restriction on the length of a type declaration: such 
a declaration can even be split over many lines, so long as its end is signaled 
by a semicolon. However, all declaration statements in a program (or program 
segment) must occur prior to the first executable statement. 

2.3 Expressions and Statements 

An expression represents a single data item — usually a number. The expression 
may consist of a single entity, such as a constant or variable, or it may consist 
of some combination of such entities, interconnected by one or more operators. 
Expressions can also represent logical conditions which are either true or false. 
However, in C, the conditions true and false are represented by the integer values 
1 and 0, respectively. Several simple expressions are given below: 



int 

double 



a, b, c; 

acc, epsilon, t; 



a + b 



x = 



y 
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t = U + V 

x <= y 

The first expression, which employs the addition operator (+), represents the sum 
of the values assigned to variables a and b. The second expression involves the 
assignment operator (=), and causes the value represented by y to be assigned 
to x. In the third expression, the value of the expression (u + v) is assigned to 
t. The fourth expression takes the value 1 (true) if the value of x is less than or 
equal to the value of y. Otherwise, the expression takes the value (false). Here, 
<= is a relational operator that compares the values of x and y. The final example 
causes the value of j to be increased by 1. Thus, the expression is equivalent to 

j = j + i 

The increment (by unity) operator ++ is called a unary operator, because it only 
possesses one operand. 

A statement causes the computer to carry out some definite action. There 
are three different classes of statements in C: expression statements, compound 
statements, and control statements. 

An expression statement consists of an expression followed by a semicolon. 
The execution of such a statement causes the associated expression to be evalu- 
ated. For example: 

a = 6; 

c = a + b; 

The first two expression statements both cause the value of the expression on 
the right of the equal sign to be assigned to the variable on the left. The third 
expression statement causes the value of j to be incremented by 1. Again, there 
is no restriction on the length of an expression statement: such a statement can 
even be split over many lines, so long as its end is signaled by a semicolon. 

A compound statement consists of several individual statements enclosed within 
a pair of braces { }. The individual statements may themselves be expression 
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statements, compound statements, or control statements. Unlike expression state- 
ments, compound statements do not end with semicolons. A typical compound 
statement is shown below: 

{ 

pi = 3.141593; 

circumference =2. * pi * radius; 
area = pi * radius * radius; 

> 

This particular compound statement consists of three expression statements, but 
acts like a single entity in the program in which it appears. 

A symbolic constant is a name that substitutes for a sequence of characters. 
The characters may represent either a number or a string. When a program is 
compiled, each occurrence of a symbolic constant is replaced by its corresponding 
character sequence. Symbolic constants are usually defined at the beginning of a 
program, by writing 

#define NAME text 

where NAME represents a symbolic name, typically written in upper-case letters, 
and text represents the sequence of characters that is associated with that name. 
Note that text does not end with a semicolon, since a symbolic constant defi- 
nition is not a true C statement. In fact, during compilation, the resolution of 
symbolic names is performed (by the C preprocessor) before the start of true com- 
pilation. For instance, suppose that a C program contains the following symbolic 
constant definition: 

#define PI 3.141593 

Suppose, further, that the program contains the statement 

area = PI * radius * radius; 

During the compilation process, the preprocessor replaces each occurrence of 
the symbolic constant PI by its corresponding text. Hence, the above statement 
becomes 

area = 3.141593 * radius * radius; 
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Symbolic constants are particularly useful in scientific programs for representing 
constants of nature, such as the mass of an electron, the speed of light, etc. Since 
these quantities are fixed, there is little point in assigning variables in which to 
store them. 

2.4 Operators 

As we have seen, general expressions are formed by joining together constants 
and variables via various operators. Operators in C fall into five main classes: 
arithmetic operators, unary operators, relational and logical operators, assignment 
operators, and the conditional operator. Let us, now, examine each of these classes 
in detail. 

There are four main arithmetic operators in C. These are: 

addition + 
subtraction 
multiplication * 
division / 

Unbelievably there is no built-in exponentiation operator in C (C was written by 
computer scientists)! Instead, there is a library function (pow) which carries out 
this operation (see later) . 

It is poor programming practice to mix types in arithmetic expressions. In 
other words, the two operands operated on by the addition, subtraction, multi- 
plication, or division operators should both be either of type int or type double. 
The value of an expression can be converted to a different data type by prepend- 
ing the name of the desired data type, enclosed in parenthesis. This type of 
construction is known as a cast. Thus, to convert an integer variable j into a 
floating-point variable with the same value, we would write 

(double) j 

Finally to avoid mixing data types when dividing a floating-point variable x by 
an integer variable i, we would write 
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x / (double) i 

Of course, the result of this operation would be of type double. 

The operators within C are grouped hierarchically according to their prece- 
dence (i.e., their order of evaluation). Amongst the arithmetic operators, * and 
/ have precedence over + and -. In other words, when evaluating expressions, 
C performs multiplication and division operations prior to addition and subtrac- 
tion operations. Of course, the rules of precedence can always be bypassed by 
judicious use of parentheses. Thus, the expression 

a - b / c + d 

is equivalent to the unambiguous expression 

a - (b / c) + d 

since division takes precedence over addition and subtraction. 

The distinguishing feature of unary operators is that they only act on single 
operands. The most common unary operator is the unary minus, which occurs 
when a numerical constant, variable, or expression is preceded by a minus sign. 
Note that the unary minus is distinctly different from the arithmetic operator 
(-) which denotes subtraction, since the latter operator acts on two separate 
operands. The two other common unary operators are the increment operator, 
++, and the decrement operator, — . The increment operator causes its operand 
to be increased by 1, whereas the decrement operator causes its operand to be 
decreased by 1. For example, — i is equivalent to i = i - 1. A cast is also 
considered to be a unary operator. Note that unary operators have precedence 
over arithmetic operators. Hence, - x + y is equivalent to the unambiguous 
expression (-x) + y, since the unary minus operator has precedence over the 
addition operator. 

Note that there is a subtle distinction between the expressions a++ and ++a. In 
the former case, the value of the variable a is returned before it is incremented. 
In the latter case, the value of a is returned after incrementation. Thus, 

b = a++; 
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is equivalent to 

b = a; 

a = a + 1; 

whereas 

b = ++a; 

is equivalent to 

a = a + 1; 
b = a; 

There is a similar distinction between the expressions a — and — a. 
There are four relational operators in C. These are: 

less than < 
less than or equal to <= 
greater than > 
greater than or equal to >= 

The precedence of these operators is lower than that of arithmetic operators. 
Closely associated with the relational operators are the two equality operators: 

equal to 

not equal to != 

The precedence of the equality operators is below that of the relational operators. 

The relational and equality operators are used to form logical expressions, 
which represent conditions that are either true or false. The resulting expressions 
are of type int, since true is represented by the integer value 1 and false by the 
integer value 0. For example, the expression i < j is true (value 1) if the value 
of i is less than the value of j, and false (value 0) otherwise. Likewise, the 
expression j == 3 is true if the value of j is equal to 3, and false otherwise. 

C also possess two logical operators. These are: 
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kk and 
I I or 

The logical operators act on operands which are themselves logical expressions. 
The net effect is to combine the individual logical expressions into more complex 
expressions that are either true or false. The result of a logical and operation is 
only true if both operands are true, whereas the result of a logical or operation 
is only false if both operands are false. For instance, the expression (i >= 5) && 
(j == 3) is true if the value of i is greater than or equal to 5 and the value of 
j is equal to 3, otherwise it is false. The precedence of the logical and operator 
is higher than that of the logical or operator, but lower than that of the equality 
operators. 

C also includes the unary operator ! that negates the value of a logical expres- 
sion: i.e., it causes an expression that is originally true to become false, and vice 
versa. This operator is referred to as the logical negation or logical not operator. 
For instance, the expression ! (k == 4) is true if the value of k is not equal to 4, 
and false otherwise. 

Note that it is poor programming practice to rely too heavily on operator prece- 
dence, since such reliance tends to makes C programs very hard for other people 
to follow. For instance, instead of writing 

i + j == 3 kk i * 1 >= 5 

and relying on the fact that arithmetic operators have precedence over relational 
and equality operators, which, in turn, have precedence over logical operators, it 
is better to write 

((i + j) == 3) kk (i * 1 >= 5) 

whose meaning is fairly unambiguous, even to people who cannot remember the 
order of precedence of the various operators in C. 

The most common assignment operator in C is =. For instance, the expression 

f = 3.4 
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causes the floating-point value 3 . 4 to be assigned to the variable f . Note that the 
assignment operator = and the equality operator == perform completely different 
functions in C, and should not be confused. Multiple assignments are permissible 
in C. For example, 

i = j = k = 4 

causes the integer value 4 to be assigned to i, j, and k, simultaneously. Note, 
again, that it is poor programming practice to mix data types in assignment ex- 
pressions. Thus, the data types of the constants or variables on either side of the 
= sign should always match. 

C contains four additional assignment operators: +=, -=, *=, and /=. The 
expression 

i += 6 

is equivalent to i = i + 6. Likewise, the expression 

i -= 6 

is equivalent to i = i - 6. The expression 

i *= 6 

is equivalent to i = i * 6. Finally, the expression 

i /= 6 

is equivalent to i = i / 6. Note that the precedence of assignment operators is 
below that of all the operators discussed previously. 

Simple conditional operations can be carried out with the conditional opera- 
tor (? :). An expression that makes use of the conditional operator is called a 
conditional expression. Such an expression takes the general form 

expression 1 ? expression 2 : expression 3 
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If expression 1 is true (i.e., if its value is nonzero) then expression 2 is eval- 
uated and becomes the value of the conditional expression. On the other hand, 
if expression 1 is false (i.e., if its value is zero) then expression 3 is evaluated 
and becomes the value of the conditional expression. For instance, the expression 

(j < 5) ? 12 : -6 

takes the value 12 if the value of j is less than 5, and the value -6 otherwise. The 
assignment statement 

k = (i < 0) ? n : m 

causes the value of n to be assigned to the variable k if the value of i is less than 
zero, and the value of m to be assigned to k otherwise. The precedence of the 
conditional operator is just above that of the assignment operators. 

As we have already mentioned, scientific programs tend to be extremely re- 
source intensive. Scientific programmers should, therefore, always be on the 
lookout for methods of speeding up the execution of their codes. It is important 
to realize that multiplication (*) and division (/) operations consume consider- 
ably more CPU time that addition (+), subtraction (-), comparison, or assignment 
operations. Thus, a simple rule of thumb for writing efficient code is to try to 
avoid redundant multiplication and division operations. This is particularly im- 
portant for sections of code which are executed repeatedly: e.g., code which lies 
within control loops. The classic illustration of this point is the evaluation of a 
polynomial. The most straightforward method of evaluating (say) a fourth-order 
polynomial would be to write something like: 

p = c_0 + c_l * x + c_2 * x * x + c_3 * x * x * x + c_4 *x*x*x*x 

Note that the above expression employs ten expensive multiplication operations. 
However, this number can be reduced to four via a simple algebraic rearrange- 
ment: 

p = c_0 + x * (c_l + x * (c_2 + x * (c_3 + x * c_4))) 

Clearly the latter expression is far more computationally efficient than the former. 
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2.5 Library Functions 

The C language is accompanied by a number of standard library functions which 
carry out various useful tasks. In particular, all input and output operations (e.g., 
writing to the terminal) and all math operations (e.g., evaluation of sines and 
cosines) are implemented by library functions. 

In order to use a library function, it is necessary to call the appropriate header 
file at the beginning of the program. The header file informs the program of the 
name, type, and number and type of arguments, of all of the functions contained 
in the library in question. A header file is called via the preprocessor statement 

#include <filename> 

where filename represents the name of the header file. 

A library function is accessed by simply writing the function name, followed 
by a list of arguments, which represent the information being passed to the func- 
tion. The arguments must be enclosed in parentheses, and separated by commas: 
they can be constants, variables, or more complex expressions. Note that the 
parentheses must be present even when there are no arguments. 

The C math library has the header file math.h, and contains the following 
useful functions: 



Function 




Type 


Purpose 


acos (d) 




double 


Return arc cosine of d (in range to pi) 


asin(d) 




double 


Return arc sine of d (in range -pi/2 to pi/2) 


atan(d) 




double 


Return arc tangent of d (in range -pi/2 to pi/2) 


atan2(dl, 


d2) 


double 


Return arc tangent of dl/d2 (in range -pi to pi) 


cbrt(d) 




double 


Return cube root of d 


cos (d) 




double 


Return cosine of d 


cosh(d) 




double 


Return hyperbolic cosine of d 


exp(d) 




double 


Return exponential of d 


fabs(d) 




double 


Return absolute value of d 


hypot (dl , 


d2) 


double 


Return sqrt(dl * dl + d2 * d2) 


log(d) 




double 


Return natural logarithm of d 


loglO(d) 




double 


Return logarithm (base 10) of d 
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pow(dl, d2) 



double 



Return dl raised to the power d2 

Return sine of d 

Return hyperbolic sine of d 

Return square root of d 

Return tangent of d 

Return hyperbolic tangent of d 



sin(d) 
sinh(d) 
sqrt (d) 
tan(d) 
tanh(d) 



double 



double 
double 
double 



double 



Here, Type refers to the data type of the quantity that is returned by the function. 
Moreover, d, dl, etc. indicate arguments of type double. 

A program that makes use of the C math library would contain the statement 

#include <math.h> 

close to its start. In the body of the program, a statement like 



would cause the variable x to be assigned a value which is the cosine of the value 
of the variable y (both x and y should be of type double). 

Note that math library functions tend to be extremely expensive in terms of CPU 
time, and should, therefore, only be employed when absolutely necessary. The 
classic illustration of this point is the use of the pow() function. This function 
assumes that, in general, it will be called with a fractional power, and, therefore, 
implements a full-blown (and very expensive) series expansion. Clearly, it is not 
computationally efficient to use this function to square or cube a quantity. In 
other words, if a quantity needs to be raised to a small, positive integer power 
then this should be implemented directly, instead of using the pow() function: i.e., 
we should write x * x rather than pow(x, 2), and x * x * x rather than pow(x, 
3), etc. (Of course, a properly designed exponentiation function would realize 
that it is more efficient to evaluate small positive integer powers by the direct 
method. Unfortunately, the pow() function was written by computer scientists!) 

The C math library comes with a useful set of predefined mathematical con- 
stants: 



x = cos(y) ; 



Name 



Description 
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M_PI 



Pi, the ratio of a circle's circumference to its diameter. 

Pi divided by two. 

Pi divided by four. 

The reciprocal of pi (1/pi) . 

The square root of two . 

The reciprocal of the square root of two 

(also the square root of 1/2) . 
The base of natural logarithms . 



M_PI_2 



M_PI_4 



M_1_PI 



M_SQRT2 
M_SQRT1_2 



M_E 



The other library functions commonly used in C programs will be introduced, 
as appropriate, during the remainder of this discussion. 

2.6 Data Input and Output 

Data input and output operations in C are carried out by the standard input/output 
library (header file: stdio.h) via the functions scanf, printf, fscanf, and 
fprintf, which read and write data from/to the terminal, and from/to a data 
file, respectively. The additional functions f open and f close open and close, re- 
spectively, connections between a C program and a data file. In the following, 
these functions are described in detail. 

The scanf function reads data from standard input (usually, the terminal). A 
call to this function takes the general form 

scanf (control_string, argl, arg2, arg3, ...) 

where control_string refers to a character string containing certain required 
formatting information, and argl, arg2, etc., are arguments that represent the 
individual input data items. 

The control string consists of individual groups of characters, with one char- 
acter group for each data input item. In its simplest form, each character group 
consists of a percent sign (°/„), followed by a set of conversion characters which 
indicate the type of the corresponding data item. The two most useful sets of 
conversion characters are as follows: 
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Character 



Type 



d 
If 



int 

double 



The arguments are a set of variables whose types match the corresponding char- 
acter groups in the control string. For reasons which will become apparent later 
on, each variable name must be preceded by an ampersand (&). Below is a typical 
application of the scanf function: 

#include <stdio.h> 

int k; 
double x, y; 

scanf ('"/.el '/.If %lf", &k, &x, &y) ; 

In this example, the scanf function reads an integer value and two floating-point 
values, from standard input, into the integer variable k and the two floating-point 
variables x and y, respectively. 

The scanf function returns an integer equal to the number of data values 
successfully read from standard input, which can be fewer than expected, or 
even zero, in the event of a matching failure. The special value EOF (which on 
most systems corresponds to —1) is returned if an end-of-file is reached before 
any attempted conversion occurs. The following code snippet gives an example 
of how the scanf function can be checked for error-free input: 

#include <stdio.h> 

int check_input; 
double x, y, z; 

check_input = scanf 07.1f '/.If '/.If", &x, &y, &z) ; 
if (check_input < 3) 
{ 

printf ("Error during data input\n") ; 



} 
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See later for an explanation of the if () construct. 

The print f function writes data to standard output (usually, the terminal). A 
call to this function takes the general form 

printf (control_string, argl, arg2, arg3, ...) 

where control_string refers to a character string containing formatting infor- 
mation, and argl, arg2, etc., are arguments that represent the individual output 
data items. 

The control string consists of individual groups of characters, with one char- 
acter group for each output data item. In its simplest form, each character group 
consists of a percent sign (%), followed by a conversion character which controls 
the format of the corresponding data item. The most useful conversion characters 
are as follows: 

Character Meaning 



d Display data item as signed decimal integer 

f Display data item as floating-point number without exponent 

e Display data item as floating-point number with exponent 

The arguments are a set of variables whose types match the corresponding char- 
acter groups in the control string (i.e., type int for d format, and type double for 
f or e format). In contrast to the scanf function, the arguments are not preceded 
by ampersands. Below is a typical application of the scanf function: 

#include <stdio.h> 
int k = 3; 

double x = 5.4, y = -9.81; 
printf ('7.d °/,f %f\n", k, x, y) ; 



In this example, the program outputs the values of the integer variable k and the 
floating-point variables x and y to the terminal. Executing the program produces 
the following output: 
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3 5.400000 -9.810000 
% 

Note that the purpose of the escape sequence \n in the control string is to gener- 
ate a line-feed after the three data items have been written to the terminal. 

Of course, the print f function can also be used to write a simple text string to 
the terminal: e.g., 

printf (text_string) 

Ordinary text can also be incorporated into the control string described above. 

An example illustrating somewhat more advanced use of the printf function 
is given below: 

#include <stdio.h> 
int k = 3; 

double x = 5.4, y = -9.81; 

printf ("k = °/„3d x + y = °/,9.4f x*y = %11.3e\n", k, x + y, x*y) ; 



Executing the program produces the following output: 

k = 3 x + y = -4.4100 x*y = -5.297e+01 

% 

Note that the final two arguments of the printf function are arithmetic expres- 
sions. Note, also, the incorporation of explanatory text into the control string. 

The character sequence °/ 3d in the control string indicates that the associated 
data item should be output as a signed decimal integer occupying a field whose 
width is at least 3 characters. More generally, the character sequence °/ nd indi- 
cates that the associated data item should be output as a signed decimal integer 
occupying a field whose width is at least n characters. If the number of charac- 
ters in the data item is less than n characters, then the data item is preceded by 
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enough leading blanks to fill the specified field. On the other hand, if the data 
item exceeds the specified field width then additional space is allocated to the 
data item, such that the entire data item is displayed. 

The character sequence °/ 9 . 4f in the control string indicates that the associated 
data item should be output as a floating-point number, in non-scientific format, 
which occupies a field of at least 9 characters, and has 4 figures after the decimal 
point. More generally, the character sequence °/ n . mf indicates that the associated 
data item should be output as a floating-point number, in non-scientific format, 
which occupies a field of at least n characters, and has m figures after the decimal 
point. 

Finally, the character sequence %H .3e in the control string indicates that the 
associated data item should be output as a floating-point number, in scientific 
format, which occupies a field of at least 11 characters, and has 3 figures after 
the decimal point. More generally, the character sequence %n . me indicates that 
the associated data item should be output as a floating-point number, in scientific 
format, which occupies a field of at least n characters, and has m figures after the 
decimal point. 

The print f function returns an integer equal to the number of printed char- 
acters, or a negative value if there was an output error. 

When working with a data file, the first step is to establish a buffer area, where 
information is temporarily stored whilst being transferred between the program 
and the file. This buffer area allows information to be read or written to the data 
file in question more rapidly than would otherwise be possible. A buffer area is 
established by writing 

FILE *stream; 

where FILE (upper-case letters required) is a special structure type that estab- 
lishes a buffer area, and stream is the identifier of the created buffer area. Note 
that a buffer area is often referred to as an input/output stream. The meaning 
of the asterisk (*) that precedes the identifier of the stream, in the above state- 
ment, will become clear later on. It is, of course, possible to establish multiple 
input/output streams (provided that their identifiers are distinct) . 
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A data file must be opened and attached to a specific input/output stream 
before it can be created or processed. This operation is performed by the function 
f open. A typical call to f open takes the form 

stream = f open(f ile_name , f ile_type) ; 

where stream is the identifier of the input/output stream to which the file is to 
be attached, and f ile_name and f ile_type are character strings that represent 
the name of the data file and the manner in which the data file will be utilized, 
respectively. The f ile_type string must be one of the strings listed below: 

file_type Meaning 



"r" Open existing file for reading only 

"w" Open new file for writing only (Any existing file 

will be overwritten) 
"a" Open existing file in append mode. (Output will be 

appended to the file) 

The f open function returns the integer value NULL (which on most systems corre- 
sponds to zero) in the event of an error. 

A data file must also be closed at the end of the program. This operation is 
performed by the function f close. The syntax for a call to f close is simply 

f close (stream) ; 

where stream is the name of the input/output stream which is to be deattached 
from a data file. The f close function returns the integer value upon successful 
completion, otherwise it returns the special value EOF. 

Data can be read from an open data file using the fscanf function, whose 
syntax is 

fscanf (stream, control_string, argl, arg2, arg3, ...) 

Here, stream is the identifier of the input/output stream to which the file is at- 
tached, and the remaining arguments have exactly the same format and meaning 
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as the corresponding arguments for the scanf function. The return values of 
f scanf are similar to those of the scanf function. 

Likewise, data can be written to an open data file using the f print f function, 
whose syntax is 

fprintf (stream, control_string, argl, arg2, arg3, ...) 

Here, stream is the identifier of the input/output stream to which the file is at- 
tached, and the remaining arguments have exactly the same format and meaning 
as the corresponding arguments for the print f function. The return values of 
fprintf are similar to those of the printf function. 

An example of a C program which outputs data to the file "data. out" is given 
below: 

#include <stdio.h> 
int k = 3; 

double x = 5.4, y = -9.81; 
FILE *output; 

output = f open ("data. out" , "w"); 
if (output == NULL) 
{ 

printf ("Error opening file data. out \n") ; 

> 

fprintf (output, "k = %3d x + y = °/ 9.4f x*y = 7,11. 3e\n", k, x + y, x*y) ; 
f close (output) ; 



On execution, the above program will write the line 

k = 3 x + y = -4.4100 x*y = -5.297e+01 

to the data file "data.out". 
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2.7 Structure of a C Program 

The syntax of a complete C program is given below: 



int main() 
{ 

return 0; 

} 

The meaning of the statements int main() and return will become clear later 
on. Preprocessor statements (e.g., #def ine and #include statements) are con- 
ventionally placed before the int main() statement. All executable statements 
must be placed between the int mainO and return statements. Function defini- 
tions (see later) are conventionally placed after the return statement. 

A simple C program (quadratic . c) that calculates the real roots of a quadratic 
equation using the well-known quadratic formula is listed below. 

/* quadratic. c */ 
/* 

Program to evaluate real roots of quadratic equation 
2 

ax + bx + c = 
using quadratic formula 

2 

x = ( -b +/- sqrt(b - 4 a c) ) / (2 a) 

*/ 

#include <stdio.h> 
#include <math.h> 

int main() 
{ 

double a, b, c, d, xl, x2; 
/* Read input data */ 



33 



2.7 Structure of a C Program 



2 SCIENTIFIC PROGRAMMING IN C 



printf ("\na = ") ; 
scanf ("%lf " , &a); 
printf ("b = ") ; 
scanf ('"/.If", &b); 
printf ("c = ") ; 
scanf C7.1f", &c); 

/* Perform calculation */ 

d = sqrt(b * b - 4. * a * c) ; 

xl = (-b + d) / (2. * a ) ; 

x2 = (-b - d) / (2. * a ); 

/* Display output */ 

printf ("\nxl = °/,12.3e x2 = 7,12. 3e\n" , xl, x2) ; 
return 0; 

} 

Note the use of comments (which are placed between the delimiters /* and */) 
to first explain the function of the program and then identify the program's major 
sections. Note, also, the use of indentation to highlight the executable statements. 
When executed, the above program produces the following output: 

a = 2 
b = 4 
c = 1 

xl = -2.929e-01 x2 = -1.707e+00 

% 

Of course, the 2, 4, and 1 were entered by the user in response to the programs's 
prompts. 

It is important to realize that there is more to writing a complete computer 
program than simply arranging the individual declarations and statements in the 
right order. Attention should also be given to making the program and its output 
as readable as possible, so that the program's function is immediately obvious to 
other people. This can be achieved by judicious use of indentation and whites- 
pace, as well as the inclusion of comments, and the generation of clearly labeled 
output. It is hoped that this approach will be exemplified by the example pro- 
grams used in this course. 
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2.8 Control Statements 

The C language includes a wide variety of powerful and flexible control state- 
ments. The most useful of these are described in the following. 

The if-else statement is used to carry out a logical test and then take one 
of two possible actions, depending on whether the outcome of the test is true or 
false. The else portion of the statement is optional. Thus, the simplest possible 
if-else statement takes the form: 

if (expression) statement 

The expression must be placed in parenthesis, as shown. In this form, the state- 
ment will only be executed if the expression has a nonzero value (i.e., if expression 
if true). If the expression has a value of zero (i.e., if expression is false) then the 
statement will be ignored. The statement can be either simple or compound. 

The program quadrat ice, listed previously, is incapable of dealing correctly 
with cases where the roots are complex (i.e., b 2 < 4 a c), or cases where a = 0. It 
is good programming practice to test for situations which fall outside the domain 
of validity of a program, and produce some sort of error message when these 
occur. An amended version of quadratic. c which uses if-else statements to 
reject invalid input data is listed below. 

/* quadratic!. . c */ 
/* 

Program to evaluate real roots of quadratic equation 
2 

ax + bx + c = 
using quadratic formula 

2 

x = ( -b +/- sqrt(b - 4 a c) ) / (2 a) 

Program rejects cases where roots are complex 
or where a = . 

*/ 

#include <stdio.h> 
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#include <math.h> 
#include <stdlib.h> 

int main() 
{ 

double a, b, c, d, e, xl, x2; 

/* Read input data */ 
printf ("\na = ") ; 
scanf ('"/.If", &a); 
printf ("b = ") ; 
scanf ("%lf", &b); 
printf ("c = ") ; 
scanf ('"/.If", &c); 

/* Test for complex roots */ 
e = b * b - 4. * a * c; 

if (e < 0.) 
{ 

printf ("\nError: roots are complex\n"); 
exit(l) ; 

} 

/* Test for a = 0. */ 
if (a == 0.) 
{ 

printf ("\nErr or: a = 0.\n"); 
exit(l) ; 

} 

/* Perform calculation */ 
d = sqrt (e) ; 

xl = (-b + d) / (2. * a ) ; 
x2 = (-b - d) / (2. * a ); 

/* Display output */ 

printf ("\nxl = °/„12.3e x2 = 7,12. 3e\n" , xl, x2) ; 
return 0; 

} 

Note the use of indentation to highlight statements which are conditionally exe- 
cuted (i.e., statements within an if-else statement). The standard library func- 
tion call exit(l) (header file: stdlib.h) causes the program to abort with an 
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error status. Execution of the above program for the case of complex roots yields 
the following output: 

a = 4 
b = 2 
c = 6 

Error: roots are complex 
% 



The general form ofanif-else statement, which includes the else clause, is 

if (expression) statement 1 else statement 2 

If the expression has a non-zero value (i.e., if expression is true) then statement 1 
is executed. Otherwise, statement 2 is executed. The program listed below is an 
extended version of the previous program quadratic . c which is capable of deal- 
ing with complex roots. 

/* quadratic2.c */ 
/* 

Program to evaluate all roots of quadratic equation 
2 

ax + bx + c = 
using quadratic formula 

2 

x = ( -b +/- sqrt(b - 4 a c) ) / (2 a) 
Program rejects cases where a = 0. 

*/ 

#include <stdio.h> 
#include <math.h> 
#include <stdlib.h> 

int main() 
{ 

double a, b, c, d, e, xl, x2; 
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/* Read input data */ 
printf ("\na = ") ; 
scanf ("°/.lf ", &a); 
printf ("b = ") ; 
scanf ("%lf", &b); 
printf ("c = ") ; 
scanf ("°/.lf", &c); 

/* Test for a = 0. */ 
if (a == 0.) 
{ 

printf ("\nError: a = 0.\n"); 
exit (1) ; 

} 

/* Perform calculation */ 
e = b * b - 4. * a * c; 

if (e > 0.) // Test for real roots 
{ 

/* Case of real roots */ 
d = sqrt (e) ; 

xl = (-b + d) / (2. * a) ; 
x2 = (-b - d) / (2. * a) ; 

printf ("\nxl = %12.3e x2 = %12.3e\n", xl, x2) ; 

} 

else 
{ 

/* Case of complex roots */ 

d = sqrt (-e) ; 

xl = -b / (2. * a ) ; 

x2 = d / (2. * a) ; 

printf ("\nxl = (7,12. 3e, 7,12. 3e) x2 = (%12.3e, %12.3e)\n" , 
xl, x2, xl, -x2) ; 

} 

return 0; 



Note the use of an if-else statement to deal with the two alternative cases of 
real and complex roots. Note also that the C compiler ignores all characters on 

a line which occur after the // construct J3 Hence, this construct can be used to 
comment individual lines in a program. The output from the above program for 

7 Strictly speaking, this is a C+ + extension to the C language. However, most modern C compilers now accept 
this comment style. 
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the case of complex roots looks like: 

a = 9 
b = 2 
c = 2 

xl = ( -l.llle-01, 4.581e-01) x2 = ( -l.llle-01, -4.581e-01) 

% 

The while statement is used to carry out looping operations, in which a group 
of statements is executed repeatedly until some condition is satisfied. The general 
form of a while statement is 

while (expression) statement 

The statement is executed repeatedly as long as the expression is nonzero (i.e., as 
long as expression is true). The statement can be either simple or compound. Of 
course, the statement must include some feature that eventually alters the value 
of the expression, thus providing an escape mechanism from the loop. 

The program listed below (iteration. c) uses a while statement to solve an 
algebraic equation via iteration, as explained in the initial comments. 

/* iteration. c */ 
/* 

Program to solve algebraic equation 



5 



2 



x + a x 



- b = 



by iteration. Easily shown that equation must have at least 
one real root. Coefficients a and b are supplied by user. 



Iteration scheme is as follows: 



2 0.2 



x 



= ( b - a x ) 



n+1 



n 



where x_n is nth iteration. User must supply initial guess for x. 
Iteration continues until relative change in x per iteration is 
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less than eps (user supplied) or until number of iterations exceeds 
NITER. Program aborts if (b - a x*x) becomes negative. 

*/ 

#include <stdio.h> 
#include <math.h> 
#include <stdlib.h> 

/* Set max. allowable no. of iterations */ 
#define NITER 30 

int main() 
{ 

double a, b, eps, x, xO, dx = 1., d; 
int count = 0; 

/* Read input data */ 
printf ("\na = ") ; 
scanf ('7„lf ", &a); 
printf ("b = ") ; 
scanf ('7„lf\ &b); 
printf ("eps = ") ; 
scanf ('"/If", &eps); 

/* Read initial guess for x */ 
printf ("\nlnitial guess for x = "); 
scanf C7,lf", &x); 
xO = x; 

while (dx > eps) // Start iteration loop: test for convergence 
{ 

/* Check for too many iterations */ 
++count ; 

if (count > NITER) 
{ 

printf ("\nError: no convergence\n") ; 
exit(l) ; 

} 

/* Reject complex roots */ 
d=b-a*x*x; 
if (d < 0.) 
{ 

printf ("Error : complex roots - try another initial guess\n"); 
exit(l) ; 
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} 

/* Perform iteration */ 
x = pow(d, 0.2); 
dx = fabs( (x - xO) / x ); 
xO = x; 

/* Output data on iteration */ 

printf ("Iter = °/„3d x = °/.8.4f dx = °/„12.3e\n", count, x, dx) ; 

} 

return 0; 

> 

The typical output from the above program looks like: 

a = 3 

b = 10 

eps = l.e-6 



Initial guess for x = 1 



Iter 




1 


X 




1. 


.4758 


dx 




3 


,224e-01 


Iter 




2 


X 




1 


,2823 


dx 




1. 


,509e-01 


Iter 




3 


X 




1. 


,3834 


dx 




7. 


,314e-02 


Iter 




4 


X 




1. 


.3361 


dx 




3, 


.541e-02 


Iter 




5 


X 




1, 


.3595 


dx 




1. 


.720e-02 


Iter 




6 


X 




1 


.3483 


dx 




8. 


,350e-03 


Iter 




7 


X 




1 


.3537 


dx 




4. 


,056e-03 


Iter 




8 


X 




1. 


.3511 


dx 




1. 


,969e-03 


Iter 




9 


X 




1. 


.3524 


dx 




9. 


,564e-04 


Iter 




10 


X 




1. 


.3518 


dx 




4 


. 644e-04 


Iter 




11 


X 




1. 


,3521 


dx 




2. 


,255e-04 


Iter 




12 


X 
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,3519 


dx 




1. 


,095e-04 


Iter 




13 


X 




1, 


.3520 


dx 




5, 


.318e-05 


Iter 




14 


X 




1. 


.3519 


dx 




2, 


.583e-05 


Iter 




15 


X 
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.3520 


dx 




1. 


,254e-05 


Iter 




16 


X 




1 


,3520 


dx 




6. 


,091e-06 


Iter 




17 


X 




1, 


.3520 


dx 




2. 


,958e-06 


Iter 




18 


X 




1. 


.3520 


dx 




1. 


.436e-06 


Iter 




19 


X 




1. 


,3520 


dx 




6. 


,975e-07 



When a loop is constructed using a while statement, the test for the continua- 
tion of the loop is carried out at the beginning of each pass. Sometimes, however, 
it is desirable to have a loop where the test for continuation takes place at the 
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end of each pass. This can be accomplished by means of a do-while statement. 
The general form of a do-while statement is 

do statement while (expression) ; 

The statement is executed repeatedly, as long as the expression is true. Note, 
however, that the statement is always executed at least once, since the test for 
repetition does not take place until the end of the first pass through the loop. 
The statement can be either simple or compound, and should, of course, include 
some feature that eventually alters the value of the expression. 

The program listed below is a marginally improved version of the previous 
program (iteration. c) which uses a do-while loop to test for convergence at 
the end (as opposed to the beginning) of each iteration loop. 

/* iterationl.c */ 
/* 

Program to solve algebraic equation 



5 



2 



x + a x 



- b = 







by iteration. Easily shown that equation must have at least 
one real root. Coefficients a and b are supplied by user. 



Iteration scheme is as follows: 



2 0.2 



x 



= ( b - a x ) 



n+1 



n 



where x_n is nth iteration. User must supply initial guess for x. 
Iteration continues until relative change in x per iteration is 
less than eps (user supplied) or until number of iterations exceeds 
NITER. Program aborts if (b - a x*x) becomes negative. 



*/ 



#include <stdio.h> 
#include <math.h> 
#include <stdlib.h> 



/* Set max. allowable no. of iterations */ 
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#define NITER 30 

int main() 
{ 

double a, b, eps, x, xO, dx, d; 
int count = 0; 

/* Read input data */ 
printf ("\na = ") ; 
scanf 07.1f", &a); 
printf ("b = ") ; 
scanf ('"/.If", &b); 
printf ("eps = ") ; 
scanf ('"/If", &eps); 

/* Read initial guess for x */ 
printf ("\nlnitial guess for x = "); 
scanf ('"/If", &x); 
xO = x; 

do // Start iteration loop 
{ 

/* Check for too many iterations */ 
++count ; 

if (count > NITER) 
{ 

printf ("\nError: no convergence\n") ; 
exit(l) ; 

} 

/* Reject complex roots */ 
d = b- a*x*x; 
if (d < 0.) 
{ 

printf ("Error : complex roots - try another initial guess\n"); 
exit(l) ; 

} 

/* Perform iteration */ 
x = pow(d, 0.2); 
dx = fabs( (x - xO) / x ); 
xO = x; 

/* Output data on iteration */ 

printf ("Iter = °/„3d x = °/.8.4f dx = %12.3e\n\ count, x, dx) ; 
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} while (dx > eps) ; // Test for convergence 
return 0; 

> 

The output from the above program is essentially identical to that from the pro- 
gram iteration. c. 

The while and do-while statements are particularly well suited to looping sit- 
uations in which the number of passes through the loop is not known in advance. 
Conversely situations in which the number of passes through the loop is known 
in advance are often best dealt with using a for statement. The general form of 
a for statement is 

for (expression 1; expression 2; expression 3) statement 

where expression 1 is used to initialize some parameter (called an index) that 
controls the looping action, expression 2 represents a condition that must be 
true for the loop to continue execution, and expressions is used to alter the 
value of the parameter initially assigned by expression 1. When a for statement 
is executed, expression 2 is evaluated and tested at the beginning of each pass 
through the loop, whereas expressions is evaluated at the end of each pass. 

The program listed below uses a for statement to evaluate the factorial of a 
non-negative integer. 

/* factorial. c */ 
/* 

Program to evaluate factorial of non-negative 
integer n supplied by user. 

*/ 

#include <stdio.h> 
#include <stdlib.h> 

int main() 
{ 

int n, count; 
double fact = 1 . ; 
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/* Read in value of n */ 
printf ("\nn = ") ; 
scanf C7.d", &n) ; 

/* Reject negative value of n */ 
if (n < 0) 
{ 

printf ("\nErr or : factorial of negative integer not defined\n") ; 
exit(l) ; 

} 

/* Calculate factorial */ 

for (count = n; count > 0; — count) fact *= (double) count; 
/* Output result */ 

printf ("\nn = %5d Factorial(n) = %12.3e\n", n, fact); 
return 0; 

} 

The typical output from the above program is shown below: 

n = 6 

n = 6 Factorial (n) = 7.200e+02 
% 

The statements which occur within if -else, while, do-while, or for state- 
ments can themselves be control statements, giving rise to the possibility of 
nested if -else statements, conditionally executed loops, nested loops, etc. When 
dealing with nested control statements, it is vital to adhere religiously to the syn- 
tax rules described above, in order to avoid confusion. 

2.9 Functions 

We have seen that C supports the use of predefined library functions which are 
used to carry out a large number of commonly occurring tasks. However, C 
also allows programmers to define their own functions. The use of programmer- 
defined functions permits a large program to be broken down into a number of 
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smaller, self-contained units. In other words, a C program can be modularized via 
the sensible use of programmer-defined functions. In general, modular programs 
are far easier to write and debug than monolithic programs. Furthermore, proper 
modularization allows other people to grasp the logical structure of a program 
with far greater ease than would otherwise be the case. 

A function is a self-contained program segment that carries out some specific, 
well-defined task. Every C program consists of one or more functions. One of 
these functions must be called main. Execution of the program always begins by 
carrying out the instructions contained in main. Note that if a program contains 
multiple functions then their definitions may appear in any order. The same 
function can be accessed from several different places within a program. Once 
the function has carried out its intended action, control is returned to the point 
from which the function was accessed. Generally speaking, a function processes 
information passed to it from the calling portion of the program, and returns 
a single value. Some functions, however, accept information but do not return 
anything. 

A function definition has two principal components: the first line (including 
the argument declarations), and the so-called body of the function. 

The first line of a function takes the general form 

data-type name (type 1 arg 1, type 2 arg 2, type n arg n) 

where data-type represents the data type of the item that is returned by the 
function, name represents the name of the function, and type 1 , type 2 , 
type n represent the data types of the arguments arg 1 , arg 2 , . . . , arg n. 
The allowable data types for a function are: 

int for a function which returns an integer value 

double for a function which returns an floating-point value 
void for a function which does not return any value 

The allowable data types for a function's arguments are int and double. Note 
that the identifiers used to reference the arguments of a function are local, in the 
sense that they are not recognized outside of the function. Thus, the argument 
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names in a function definition need not be the same as those used in the segments 
of the program from which the function was called. However, the corresponding 
data types of the arguments must always match. 

The body of a function is a compound statement that defines the action to 
be taken by the function. Like a regular compound statement, the body can 
contain expression statements, control statements, other compound statements, 
etc. The body can even access other functions. In fact, it can even access itself — 
this process is known as recursion. In addition, however, the body must include 
one or more return statements in order to return a value to the calling portion 
of the program. 

A return statement causes the program logic to return to the point in the 
program from which the function was accessed. The general form of a return 
statement is: 

return expression; 

This statement causes the value of expression to be returned to the calling part of 
the program. Of course, the data type of expression should match the declared 
data type of the function. For a void function, which does not return any value, 
the appropriate return statement is simply: 

return ; 

A maximum of one expression can be included in a return statement. Thus, a 
function can return a maximum of one value to the calling part of the program. 
However, a function definition can include multiple return statements, each con- 
taining a different expression, which are conditionally executed, depending on 
the program logic. 

Note that, by convention, the main function is of type int and returns the 
integer value to the operating system, indicating the error-free termination of 
the program. In its simplest form, the main function possesses no arguments. The 
library function call exit (1), employed in previous example programs, causes the 
execution of a program to abort, returning the integer value 1 to the operating 
system, which (by convention) indicates that the program terminated with an 
error status. 
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The program segment listed below shows how the previous program factorial . c 
can be converted into a function factorial (n) which returns the factorial (in the 
form of a floating-point number) of the non-negative integer n: 

double factorial (int n) 
{ 

/* 

Function to evaluate factorial (in floating-point form) 
of non-negative integer n. 

*/ 

int count ; 
double fact = 1 . ; 

/* Abort if n is negative integer */ 
if (n < 0) 
{ 

printf ("\nError : factorial of negative integer not defined\n") ; 
exit(l) ; 

} 

/* Calculate factorial */ 

for (count = n; count > 0; — count) fact *= (double) count; 

/* Return value of factorial */ 
return fact; 

} 

A function can be accessed, or called, by specifying its name, followed by a 
list of arguments enclosed in parentheses and separated by commas. If the func- 
tion call does not require any arguments then an empty pair of parentheses must 
follow the name of the function. The function call may be part of a simple expres- 
sion, such as an assignment statement, or it may be one of the operands within 
a more complex expression. The arguments appearing in a function call may be 
expressed as constants, single variables, or more complex expressions. However, 
both the number and the types of the arguments must match those in the function 
definition. 

The program listed below (printf act . c) uses the function factorialO, de- 
scribed above, to print out the factorials of all the integers between and 20: 
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/* printfact.c */ 
/* 

Program to print factorials of all integers 
between and 20 

*/ 

#include <stdio.h> 
#include <stdlib.h> 

/ /0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/ 

/ / /o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/oA 

double factorial (int n) 

{ 

/* 

Function to evaluate factorial (in floating-point form) 
of non-negative integer n. 

*/ 

int count ; 
double fact = 1 . ; 

/* Abort if n is negative integer */ 
if (n < 0) 
{ 

printf ("\nError : factorial of negative integer not defined\n") ; 
exit(l) ; 

} 

/* Calculate factorial */ 

for (count = n; count > 0; — count) fact *= (double) count; 

/* Return value of factorial */ 
return fact; 

> 

/ /0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/ 

/ / /o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/oA 

int main() 

{ 

int j ; 

/* Print factorials of all integers between and 20 */ 
for (j = 0; j <= 20; ++j) 

printf ("j = %3d factorial(j) = %12.3e\n", j, f actorial ( j ) ) ; 
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return 0; 

} 

Note that the call to factorial () takes place inside a complex expression (i.e., 
a printf () function call). Note also that the argument of factorial () has a 
different name (but the same data type) in the two sections of the program (i.e., 
in the main() function and the factorial () function). The output from the 
above program looks like: 



j 




n 


IdC LOI idl U ^ — 


± 




j 




1 


■f a rtr^yi a 1 f i 1 — 
J. CLO l/U-L XcLJ. V, J ) — 


± 




J 




o 


"Fa^"l~riT"ial i i 1 — 
J. do uui 1 al \ J / 


9 




j 
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factorial (j) = 


6 


000e+00 


j 




4 


factorial (j) = 


2 


400e+01 


j 




5 


factorial (j) = 


1 


200e+02 


j 




6 


factorial (j) = 


7 


200e+02 


j 




7 


factorial (j) = 


5 


040e+03 


j 




8 


factorial (j) = 


4 


032e+04 


j 




9 


factorial (j) = 


3 


629e+05 


j 




10 


factorial (j) = 


3 


629e+06 


j 




11 


factorial (j) = 


3 


992e+07 


j 




12 


factorial (j) = 


4 


790e+08 


j 




13 


factorial (j) = 


6 


227e+09 


j 




14 


factorial (j) = 


8 


718e+10 


j 




15 


factorial (j) = 


1 


308e+12 


j 




16 


factorial (j) = 


2 


092e+13 


j 




17 


factorial (j) = 


3 


557e+14 


j 




18 


factorial (j) = 


6 


402e+15 


j 




19 


factorial (j) = 


1 


216e+17 


j 




20 


factorial (j) = 


2 


433e+18 



% 

Ideally, function definitions should always precede the corresponding function 
calls in a C program. This requirement can usually be satisfied by judicious or- 
dering of the various functions which make up a program, but, almost inevitably, 
restricts the location of the main() function to the end. of the program. Hence, 
if the order of the two functions in the above program [i.e., factorial () and 
mainO] were swapped then an error message would be generated on compila- 
tion, since an attempt would be made to call factorial () prior to its definition. 
Unfortunately, for the sake of logical clarity, most C programmers prefer to place 
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the main() function at the beginning of their programs. After all, main() is al- 
ways the first part of a program to be executed. In such situations, function calls 
[within mainO ] are bound to precede the corresponding function definitions: for- 
tunately, however, compilation errors can be avoided by using a construct known 
as a function prototype. 

Function prototypes are conventionally placed at the beginning of a program 
(i.e., before the mainO function) and are used to inform the compiler of the 
name, data type, and number and data types of the arguments, of all user-defined 
functions employed in the program. The general form of a function prototype is 

data-type name (type 1, type 2, type n) ; 

where data-type represents the data type of the item returned by the referenced 
function, name is the name of the function, and type 1, type 2, . . . , type n 
are the data types of the arguments of the function. Note that it is not necessary 
to specify the names of the arguments in a function prototype. Incidentally, the 
function prototypes for predefined library functions are contained within the as- 
sociated header files which must be included at the beginning of every program 
which uses these functions. 

The program listed below is a modified version of print fact . c in which the 
mainO function is the first function to be defined: 

/* printfactl.c */ 
/* 

Program to print factorials of all integers 
between and 20 

*/ 

#include <stdio.h> 
#include <stdlib.h> 

/* Prototype for function factorial () */ 
double f actorial(int) ; 

int main() 
{ 

int j; 



51 



2.9 Functions 



2 SCIENTIFIC PROGRAMMING IN C 



/* Print factorials of all integers between and 20 */ 
for (j = 0; j <= 20; 

printf ("j = %3d factorial(j) = 7ol2.3e\n", j, f actorial ( j ) ) ; 

return 0; 

} 

/ /V V °/ °/ °/ / °/V V V V V °/ °/V V V °/°/°/V V V V V V °/V V °/ 0/0/0/0/0/0/ '/ 9/o/o/oyoyo/ 0/ 0/o/o/oy 0/0/ 0/0/ 0/ 0/0/0/0/0/0/ 
/ / /o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/oA 

double f actorial (int n) 

{ 

/* 

Function to evaluate factorial (in floating-point form) 
of non-negative integer n. 

*/ 



int count ; 
double fact = 1 . ; 



/* Abort if n is negative integer */ 
if (n < 0) 
{ 

printf ("\nError : factorial of negative integer not def ined\n") ; 
exit (1) ; 

} 

/* Calculate factorial */ 

for (count = n; count > 0; — count) fact *= (double) count; 



/* Return value of factorial */ 
return fact; 

> 



Note the presence of the function prototype for factorial () prior to the defi- 
nition of main(). This is needed because the program calls factorial () before 
this function has been defined. The output from the above program is identical 
to that from printf act . c. 

It is generally considered to be good programming practice to provide function 
prototypes for all user-defined functions accessed in a program, whether or not 
they are strictly required by the compiler]^ The reason for this is fairly simple. 



3 Function prototypes are a requirement for all user-defined functions in C+ + programs. 
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If we provide a prototype for a given function then the compiler can carefully 
compare each use of the function, within the program, with this prototype so as 
to determine whether or not we are calling the function properly. In the absence 
of a prototype, an incorrect call to a function (e.g., using the wrong number of 
arguments, or arguments of the wrong data type) can give rise to run-time errors 
which are difficult to diagnose. 

When a single value is passed to a function as an argument then the value of 
that argument is simply copied to the function. Thus, the argument's value can 
subsequently be altered within the function but this will not affect its value in the 
calling routine. This procedure for passing the value of an argument to a function 
is called passing by value. 

Passing an argument by value has advantages and disadvantages. The advan- 
tages are that it allows a single-valued argument to be written as an expression, 
rather than being restricted to a single variable. Furthermore, in cases where 
the argument is a variable, the value of this variable is protected from alterations 
which take place within the function. The main disadvantage is that information 
cannot be transferred back to the calling portion of the program via arguments. 
In other words, passing by value is a strictly one-way method of transferring 
information. The program listed below, which is another modified version of 
print fact . c, illustrates this point: 

/* printfact2.c */ 
/* 

Program to print factorials of all integers 
between and 20 

*/ 

#include <stdio.h> 
#include <stdlib.h> 

/* Prototype for function factorial () */ 
double f actorial(int) ; 

int main() 
{ 

int j ; 

/* Print factorials of all integers between and 20 */ 
for (j = 0; j <= 20; ++j) 
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printfO'j = 7„3d factorial(j) = 7,12. 3e\n" , j, factorial(j)) ; 
return 0; 

} 

/ /0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/ 

/ / /o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/oA 

double factorial (int n) 

{ 

/* 

Function to evaluate factorial (in floating-point form) 
of non-negative integer n. 

*/ 

double fact = 1 . ; 

/* Abort if n is negative integer */ 
if (n < 0) 
{ 

printf ("\nError : factorial of negative integer not defined\n") ; 
exit(l) ; 

} 

/* Calculate factorial */ 

for (; n > 0; — n) fact *= (double) n; 

/* Return value of factorial */ 
return fact; 

> 



Note that the function factorial () has been modified such that its integer ar- 
gument n is also used as the index in a for statement. Thus, the value of this 
argument is modified within the function. Nevertheless, the output of the above 
program is identical to that from printf act. c, since the modifications to n are 
not passed back to the main part of the program. Note, incidentally, the use of a 
null initialization expression in the for statement appearing in factorial (). 
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2.10 Pointers 

One of the main characteristics of a scientific program is that large amounts of 
numerical information are exchanged between the various functions which make 
up the program. It is generally most convenient to pass this information via the 
argument lists, rather than the names, of the these functions. After all, only one 
number can be passed via a function name, whereas scientific programs generally 
require far more than one number to be passed during a function call. Hence, 
the functions employed in scientific programs generally return no values via their 
names (i.e., they tend to be of data type void) but possess large strings of argu- 
ments. There is one obvious problem with this approach. Namely, a void function 
which passes all of its arguments by value is incapable of returning any informa- 
tion to the program segment from which it was called. Fortunately, there is a 
way of getting around this difficulty: we can pass the arguments of a function by 
reference, rather than by value, using constructs known as pointers. This allows 
the two-way communication of information via arguments during function calls. 
Pointers are discussed in the following. 

Suppose that v is a variable in a C program which represents some particular 
data item. Of course, the program stores this data item at some particular location 
in the computer's memory. The data item can thus be accessed if we know its 
location, or address, in computer memory. The address of v's memory location 
is determined by the expression &v, where & is a unary operator known as the 
address operator. 

Suppose that we assign the address of v to another variable pv. In other words, 

pv = &v 

This new variable is called a pointer to v, since it points to the location where v 
is stored in memory. Remember, however, that pv represents v's address, and not 
its value. 

The data item represented by v (i.e., the data item stored at v's memory loca- 
tion) can be accessed via the expression *pv, where * is a unary operator, called 
the indirection operator, which only operates on pointer variables. Thus, *pv 
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and v both represent the same data item. Furthermore, if we write pv = &v and 
u = *pv then both u and v represent the same value. 

The simple program listed below illustrates some of the points made above: 

/* pointer. c */ 
/* 

Simple illustration of the action of pointers 

*/ 

#include <stdio.h> 

main() 
{ 



int 


u = 5; 




int 


v; 




int 


*pu; 


// Declare pointer to an integer variable 


int 


*pv; 


// Declare pointer to an integer variable 


pu = 


= &u; 


// Assign address of u to pu 


v = 


*pu; 


// Assign value of u to v 


pv = 


= &v; 


// Assign address of v to pv 



printf("\nu = %d &u = °/„X pu = °/„X *pu = %d" , u, &u, pu, *pu) ; 
printf("\nv = %d &v = °/,X pv = °/„X *pv = °/„d\n" , v, &v, pv, *pv) ; 

return 0; 

> 

Note that pu is a pointer to u, whereas pv is a pointer to v. Incidentally, the con- 
version character X, which appears in the control strings of the above printf () 
function calls, indicates that the associated data item should be output as a hex- 
adecimal number — this is the conventional method of representing an address in 
computer memory. Execution of the above program yields the following output: 

u = 5 &u = BFFFFA24 pu = BFFFFA24 *pu = 5 
v = 5 &v = BFFFFA20 pv = BFFFFA20 *pv = 5 
% 

In the first line, we see that u represents the value 5, as specified in its declaration 
statement. The address of u is determined automatically by the compiler to be 
BFFFFA24 (hexadecimal). The pointer pu is assigned this value. Finally, the value 
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to which pu points is 5, as expected. Similarly, the second line shows that v also 
represents the value 5. This is as expected, since we assigned the value *pu to v. 
The address of v is BFFFFA20. Of course, u and v have different addresses. 

The unary operators & and * are members of the same precedence group as 
the other unary operators (e.g., ++ and — ). The address operator (&) can only act 
upon operands which possess a unique address, such as ordinary variables. Thus, 
the address operator cannot act upon arithmetic expression, such as 2 * (u + v). 
The indirection operator (*) can only act upon operands which are pointers. 

Pointer variables, like all other variables, must be declared before they can 
appear in executable statements. A pointer declaration takes the general form 



data-type *ptvar; 



where ptvar is the name of the pointer variable, and data-type is the data type 
of the data item towards which the pointer points. Note that an asterisk must 
always precede the name of a pointer variable in a pointer declaration. 



Referring to Sect. 12. 6[ we can now appreciate that the mysterious asterisk 



which appears in the declaration of an input/output stream, e.g., 



FILE *stream; 



is there because stream is a pointer variable (pointing towards an object of the 
special data type FILE). In fact, stream points towards the beginning of the asso- 
ciated input/output stream in memory. 

Pointers are often passed to a function as arguments. This allows data items 
within the calling part of the program to be accessed by the function, altered 
within the function, and then passed back to the calling portion of the program in 
altered form. This use of pointers is referred to as passing arguments by reference, 
rather than by value. 

When an argument is passed by value, the associated data item is simply 
copied to the function. Thus, any alteration to the data item within the func- 
tion is not passed back to the calling routine. When an argument is passed by 
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reference, however, the address of the associated data item is passed to the func- 
tion. The contents of this address can be freely accessed by both the function and 
the calling routine. Furthermore, any changes made to the data item stored at 
this address are recognized by both the function and the calling routine. Thus, 
the use of a pointer as an argument allows the two-way communication of infor- 
mation between a function and its calling routine. 

The program listed below, which is yet another modified version of print fact . c, 
uses a pointer to pass back information from a function to its calling routine: 

/* printfact3.c */ 
/* 

Program to print factorials of all integers 
between and 20 

*/ 

#include <stdio.h> 
#include <stdlib.h> 

/* Prototype for function factorial () */ 
void f actorial(int , double *) ; 

int main() 
{ 

int j; 

double fact; 

/* Print factorials of all integers between and 20 */ 
for (j = 0; j <= 20; ++j) 
{ 

factorial (j , &f act) ; 

printf("j = %3d factorial(j) = %12.3e\n", j, fact); 

} 

return 0; 

> 

/ /<y o/o/ o/o/ o/o/ o/o/ o/o/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ hi hi 0/0/ a/y 0/0/0/0/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/oy 0/0/ 0/0/0/0/ 
/ / /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o A 

void f actorial(int n, double *fact) 
{ 

/* 

Function to evaluate factorial *fact (in floating-point form) 
of non-negative integer n. 
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*/ 

*f act = 1 . ; 

/* Abort if n is negative integer */ 
if (n < 0) 
{ 

printf ("\nError : factorial of negative integer not defined\n") ; 
exit(l) ; 

} 

/* Calculate factorial */ 

for (; n > 0; — n) *fact *= (double) n; 

return; 

> 

The output from this program is again identical to that from printf act . c. Note 
that the function factorial () has been modified such that there is no data item 
associated with its name (i.e., the function is of data type void). However, the 
argument list of this function has been extended such that there are now two 
arguments. As before, the first argument is the value of the positive integer n 
whose factorial is to be evaluated by the function. The second argument, fact, 
is a pointer which passes back the factorial of n (in the form of a floating-point 
number) to the main part of the program. Incidentally, the compiler knows that 
fact is a pointer because its name is proceeded by an asterisk in the argument 
declaration for factorial (). Of course, in the body of the function, reference 
is made to *f act (i.e., the value of the data item stored in the memory location 
towards which fact points) rather than fact (i.e., the address of the memory 
location towards which fact points). Note that a void function, which returns 
no value, can only be called via a statement consisting of the function name 
followed by a list of its arguments (in parentheses and separated by commas). 
Thus, the function factorial () is called in the main part of the program via the 
statement 

factoriaKj, &f act) ; 

This statement passes the integer value j to factorial (), which, in turn, passes 
back the value of the factorial of j via its second argument. Note that since the 
second argument is passed by reference, rather than by value, it is written &f act 
(i.e., the address of the memory location where the floating-point value fact is 
stored) rather than fact (i.e., the value of the floating-point variable fact). Note, 
finally, that the function prototype for factorial () takes the form 
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void f actor ial(int , double *) ; 

Here, the asterisk after double indicates that the second argument is a pointer to 
a floating-point data item. 

We can now appreciate that the mysterious ampersands which must precede 
variable names in scanf () calls: e.g., 

scanf ("%d '/.If %lf", &k, &x, &y) ; 

are not so mysterious, after all. scanf () is a function which returns data to its 
calling routine via its arguments (excluding its first argument, which is a control 
string). Hence, these arguments must be passed to scanf () by reference, rather 
than by value, otherwise they would be unable to pass information back to the 
calling routine. It follows that we must pass the addresses of variables (e.g., &k) 
to scanf (), rather than the values of these variables (e.g., k). Note that since the 
print f () function does not return any information to its calling routine via its 
arguments, there is no need to pass these arguments by reference — passing by 
value is fine. This explains why there are no ampersands in the argument list of 
a print f () function. 

A pointer to a function can be passed to another function as an argument. This 
allows one function to be transferred to another, as though the first function were 
a variable. This is very useful in scientific programming. Imagine that we have a 
routine which numerically integrates a general one-dimensional function. Ideally, 
we would like to use this routine to integrate more than one specific function. 
We can achieve this by passing (to the routine) the name of the function to be 
integrated as an argument. Thus, for example, we can use the same routine to 
integrate a polynomial, a trigonometric function, or a logarithmic function. 

Let us refer to the function whose name is passed as an argument as the guest 
function. Likewise, the function to which this name is passed is called the host 
function. A pointer to a guest function is identified in the host function definition 
by an entry of the form 

data-type (*f unction-name) (type 1, type 2, ...) 



60 



2.10 Pointers 



2 SCIENTIFIC PROGRAMMING IN C 



in the host function's argument declaration^ Here, data-type is the data type 
of the guest function, function-name is the local name of the guest function in 
the host function definition, and type 1 , type 2, ... are the data types of the 
guest function's arguments. The pointer to the guest function also requires an 
entry of the form 

data-type (*) (type 1, type 2, ...) 

in the argument declaration of the host function's prototype. The guest function 
can be accessed within the host function definition by means of the indirection 
operator. To achieve this, the indirection operator must precede the guest func- 
tion name, and both the indirection operator and the guest function name must 
be enclosed in parenthesis: i.e., 

(*f unction-name) (arg 1, arg 2, ...) 

Here, arg 1 , arg 2, . . . are the arguments passed to the guest function. Finally, 
the name of a guest function is passed to the host function, during a call to the 
latter function, via an entry like 

function-name 

in the host function's argument list. 

The program listed below is a rather silly example which illustrates the passing 
of function names as arguments to another function: 

/* passf unction. c */ 
/* 

Program to illustrate the passing of function names as 
arguments to other functions via pointers 

*/ 

#include <stdio.h> 

/* Function prototype for host fun. */ 

void cube (double (*) (double) , double, double *) ; 

9 The parenthesis around *f unction-name are very important: 
data-type *f unction-name (type 1, type 2, . . .) is interpreted by the compiler as a reference to a function 
which returns a pointer to type data-type, rather than a pointer to a function which returns type data-type. 
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double funl (double) ; // Function prototype for first guest function 
double fun2(double) ; // Function prototype for second guest function 

int main() 
{ 

double x, resl, res2; 

/* Input value of x */ 
printf ("\nx = ") ; 
scanf ('"/.If", &x); 

/* Evaluate cube of value of first guest function at x */ 
cube (funl, x, feresl) ; 

/* Evaluate cube of value of second guest function at x */ 
cube(fun2, x, &res2) ; 

/* Output results */ 

printf ("\nx = %8.4f resl = %8.4f res2 = °/,8.4f\n", x, resl, res2) ; 
return 0; 

} 

/ /o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/ 
/ / /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o A 

void cube(double (*fun) (double) , double x, double *result) 
{ 

/* 

Host function: accepts name of floating-point guest function 
with single floating-point argument as its first argument, 
evaluates this function at x (the value of its second argument) , 
cubes the result, and returns final result via its third argument. 

*/ 

double y; 

y = (*fun) (x) ; // Evaluate guest function at x 

*result = y * y * y; // Cube value of guest function at x 

return; 

> 

/ /V V V V V V V V V V V V V V V V V V V V V V V V °/ V V V V V V V V V V V V °/ °/ V V V V V °/ V V V V / / / / / / / / / / 
/ / /o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o A 
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double f unl (double z) 
{ 

/* 

First guest function 

*/ 

return 3.0*z*z-z; 

> 

/ /0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/ 

/ / /o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/oA 

double fun2 (double z) 

{ 

/* 

Second guest function 

*/ 

return 4.0 * z -5.0*z*z*z; 

} 

In the above program, the function cubeO accepts the name of a guest function 
(with one argument) as its first argument, evaluates this function at x (the value 
of its second argument, which is ultimately specified by the user), cubes the re- 
sult, and then passes the final result back to the main part of the program via its 
third argument (which, of course, is a pointer). The two guest functions, funl() 
and fun2(), whose names are passed to cubeO, are both simple polynomials. 
The output from the above program looks like: 

x = 2 

x = 2.0000 resl = 1000.0000 res2 = -32768.0000 



2.11 Global Variables 

We have seen that a general C program is basically a collection of functions. Fur- 
thermore, the variables used by these functions are local in scope: i.e., a variable 
defined in one function is not recognized in another. The main method of trans- 
ferring data from one function to another is via the argument lists in function 
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calls. Arguments can be passed in one of two different manners. When an ar- 
gument is passed by value then the value of a local variable (or expression) in 
the calling routine is copied to a local variable in the function which is called. 
When an argument is passed by reference then a local variable in the calling rou- 
tine shares the same memory location as a local variable in the function which 
is called: hence, a change in one variable is automatically mirrored in the other. 
However, there is a third method of transferring information from one function 
to another. It is possible to define variables which are global in extent: such vari- 
ables are recognized by all the functions making up the program, and have the 
same values in all of these functions. 

The C compiler recognizes a variable as global, as opposed to local, because 
its declaration is located outside the scope of any of the functions making up the 
program. Of course, a global variable can only be used in an executable state- 
ment after it has been declared. Hence, the natural place to put global variable 
declaration statements is before any function definitions: i.e., right at the begin- 
ning of the program. Global variables declarations can be used to initialize such 
variables, in the regular manner. However, the initial values must be expressed 
as constants, rather than expressions. Furthermore, the initial values are only 
assigned once, at the beginning of the program. 

The program listed below is yet another version of printf act . c, in which 
communication between the two sections of the program takes place entirely via 
global variables: 

/* printf act4.c */ 
/* 

Program to print factorials of all integers 
between and 20 

*/ 

#include <stdio.h> 
#include <stdlib.h> 

/* Prototype for function factorial () */ 
void factorialQ; 

/* Global variable declarations */ 
int j; 
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double fact; 

int main() 
{ 

/* Print factorials of all integers between and 20 */ 
for (j = 0; j <= 20; ++j) 
{ 

factorial () ; 

printf("j = %3d factorial(j) = 7,12. 3e\n" , j, fact); 

} 

return 0; 

> 

/ /o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ o/o/ 0/0/ 0/0/ 0/0/ 0/0/ o/o /o/o/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ op/ 0/0/ 
/ I /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o A 

void factorialQ 
{ 

/* 

Function to evaluate factorial (in floating-point form) 
of non-negative integer j. Result stored in variable fact. 

*/ 

int count ; 

/* Abort if j is negative integer */ 
if (j < 0) 
{ 

printf ("\nError : factorial of negative integer not defined\n") ; 
exit(l) ; 

} 

/* Calculate factorial */ 

for (count = j, fact =1.; count > 0; — count) fact *= (double) count; 
return; 

} 

The output from the above program is identical to that from printf act . c. Ob- 
serve that the global variable j is used to pass the integer value whose factorial is 
to be calculated from the main part of the program to the function factorial (), 
whilst the global variable fact is used to pass the calculated factorial back to 
the main part of the program. Incidentally, note the use of multiple initial- 
ization statements (separated by commas) in the for statement appearing in 
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factorial (). 

Global variables should be used sparingly in scientific programs (or any other 
type of program), since there are inherent dangers in their employment. An 
alteration in the value of a global variable within a given function is carried 
over into all other parts of the program. Unfortunately such an alteration can 
sometimes happen inadvertently as the side-effect of some other action. Thus, 
there is the possibility of the value of a global variable changing unexpectedly, 
resulting in a subtle programming error which can be extremely difficult to track 
down, since the offending line could be located anywhere in the program. Similar 
errors involving local variables are much easier to debug, since the scope of local 
variables is far more limited than that of global variables. 

2.12 Arrays 

Scientific programs very often deal with multiple data items possessing common 
characteristics. In such cases, it is often convenient to place the data items in 
question into an array, so that they all share a common name (e.g., x). The 
individual data items can be either integers or floating-point numbers. However, 
they all must be of the same data type. 

In C, an element of an array (i.e., an individual data item) is referred to by 
specifying the array name followed by one or more subscripts, with each sub- 
script enclosed in square brackets. All subscripts must be nonnegative integers. 
Thus, in an n-element array called x, the array elements are x[0] , x[l] , 
x [n-1] . Note that the first element of the array is x [0] and not x [1] , as in other 
programming languages. 

The number of subscripts determines the dimensionality of an array. For exam- 
ple, x[i] refers to an element of a one-dimensional array, x. Similarly, y[i] [j] 
refers to an element of a two-dimensional array, y, etc. 

Arrays are declared in much the same manner as ordinary variables, except 
that each array name must be accompanied by a size specification (which speci- 
fies the number of elements) . For a one-dimensional array, the size is specified by 
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a positive integer constant, enclosed in square brackets. The generalization for 
multi-dimensional arrays is fairly obvious. Several valid array declarations are 
shown below: 

int j [100] ; 
double x [20] ; 
double y[10] [20] ; 

Thus, j is a 100-element integer array, x is a 20-element floating point array, and 
y is a 10x20 floating-point array. Note that variable size array declarations, e.g., 

double a[n] ; 

where n is an integer variable, are illegal in C. 

It is sometimes convenient to define an array size in terms of a symbolic con- 
stant, rather than a fixed integer quantity. This makes it easier to modify a pro- 
gram that utilizes an array, since all references to the maximum array size can be 
altered by simply changing the value of the symbolic constant. This approach is 
used in many of the example programs employed in this course. 

Like an ordinary variable, an array can be either local or global in extent, 
depending on whether the associated array declaration lies inside or outside, 
respectively, the scope of any of the functions which constitute the prograrrLBoth 
local and global arrays can be initialized via their declaration statements]^ For 
instance, 

int j [5] = {1, 3, 5, 7, 9}; 

declares j to be a 5-element integer array whose elements have the initial values 
j[0]=l, j[l]=3, etc. 

Single operations which involve entire arrays are not permitted in C. Thus, if x 
and y are similar arrays (i.e., the same data type, dimensionality, and size) then 
assignment operations, comparison operations, etc. involving these two arrays 

10 Prior to the ANSI standard, to which the GNU C compiler adheres, local arrays could not be initialized via their 
declaration statements. 
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must be carried out on an element by element basis. This is usually accomplished 
within a loop (or within nested loops, for multi-dimensional arrays). 

The program listed below is a simple illustration of the use of arrays in C. 
The program reads a list of numbers, entered by the user, into a one-dimensional 
array, list, and then calculates the average of these numbers. The program also 
calculates and outputs the deviation of each number from the average. 

/* aver age. c */ 
/* 

Program to calculate the average of n numbers and then 
compute the deviation of each number from the average 

Code adapted from "Programming with C" , 2nd Edition, Byron Gottfreid, 
Schaum's Outline Series, (McGraw-Hill, New York NY, 1996) 

*/ 

#include <stdio.h> 
#include <stdlib.h> 

#define NMAX 100 

int main() 
{ 

int n, count; 

double avg, d, sum =0.; 

double list [NMAX] ; 

/* Read in value for n */ 

printf ("\nHow many numbers do you want to average? "); 
scanf C7,d", &n) ; 

/* Check that n is not too large or too small */ 
if ((n > NMAX) | | (n <= 0)) 
{ 

printf ("\nErr or : invalid value for n\n"); 
exit(l) ; 

} 

/* Read in the numbers and calculate their sum */ 
for (count = 0; count < n; ++count) 
{ 

printf ("i = %d x = ", count +1); 
scanf C7„lf", felist [count] ) ; 
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sum += list [count] ; 

} 

/* Calculate and display the average */ 
avg = sum / (double) n; 

printf ("\nThe average is %5 . 2f \n\n" , avg); 

/* Calculate and display the deviations about the average */ 
for (count = 0; count < n; ++count) 
{ 

d = list [count] - avg; 

printf ("i = %d x = °/,5.2f d = °/,5.2f\n", count + 1, list [count], d) ; 

} 

return 0; 



Note the use of the symbolic constant MAX to specify the size of the array list, 
and, hence, the maximum number of values which can be averaged. The typical 
output from the above program looks like: 

How many numbers do you want to average? 5 



i = 


1 


x = 4.6 








i = 


2 


x = -2.3 








i = 


3 


x = 8.7 








i = 


4 


x = 0.12 








i = 


5 


x = -2.7 








The 


average is 1 


.68 






i = 


1 


x = 4.60 


d = 


2, 


,92 


i = 


2 


x = -2.30 


d = 


-3. 


.98 


i = 


3 


x = 8.70 


d = 


7, 


,02 


i = 


4 


x = 0.12 


d = 


-1. 


.56 


i = 


5 


x = -2.70 


d = 


-4. 


,38 



7. 



It is important to realize that an array name in C is essentially a pointer to the 
first element in that arrayo Thus, if x is a one-dimensional array then the address 
of the first array element can be expressed as either &x [0] or simply x. Moreover, 

n An array name is not exactly equivalent to a pointer, because a pointer can point to any address in the computer 
memory — this address can even be changed — whereas an array name is constrained to always point towards the 
address of its associated first data item. 
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the address of the second array element can be written as either &x [1] or (x+1) . 
In general, the address of the (i+l)th array element can be expressed as either 
&x [i] or (x+i) . Incidentally it should be understood that (x+i) is a rather special 
type of expression, since x represents an address, whereas i represents an integer 
quantity. The expression (x+i) actually specifies the address of the array element 
which is i memory locations offset from the address of the first array element (C, 
of course, stores all elements of an array both contiguously and in order in the 
computer memory). Hence, (x+i) is a symbolic representation of an address, 
rather than an arithmetic expression. 

Since &x[i] and (x+i) both represent the address of the (i+l)th element of 
the array x, it follows that x[i] and *(x+i) must both represent the contents of 
that address (i.e., the value of the (i+l)th element). In fact, the latter two terms 
are completely interchangeable in C programs. 

For the moment, let us concentrate on one-dimensional arrays. An entire array 
can be passed to a function as an argument. To achieve this, the array name 
must appear by itself, without brackets or subscripts, as an argument within the 
function call. The corresponding argument in the function definition must be 
declared as an array. In order to do this, the array name is written followed by 
an empty pair of square brackets. The size of the array is not specified. In a 
function prototype, an array argument is specified by following the data type of 
the argument by an empty pair of square brackets. 

Since, as we have seen, an array name is essentially a pointer, it is clear that 
when an array is passed to a function it is passed by reference, and not by value. 
Hence, if any of the array elements are altered within the function then these 
alterations are recognized in the calling portion of the program. Likewise, if 
an array (rather than an individual array element) appears in the argument list 
of a scanf () function then it should not be preceded by the address operator 
(&), since an array name already is an address. The reason why arrays in C are 
always passed by reference is fairly obvious. In order to pass an array by value, 
it is necessary to copy the value of every element. On the other hand, to pass an 
array by reference it is only necessary to pass the address of the first element. 
Clearly, for large arrays, passing by reference is far more efficient than passing by 



70 



2.12 Arrays 



2 SCIENTIFIC PROGRAMMING IN C 



value. 



The program listed below is yet another version of print fact . c, albeit a far 
more efficient one than any of those listed previously. In this version, the factori- 
als of all the non-zero integers up to 20 are calculated in one fell swoop, by the 
function factorial (), using the recursion relation 



The factorials are stored as elements of the array fact [], which is passed as an 
argument from factorial () to the main part of the program. 

/* printfact5.c */ 
/* 

Program to print factorials of all integers 
between and 20 

*/ 

#include <stdio.h> 

/* Function prototype for factorial () */ 
void factorial (double [] ) ; 



/* Calculate factorials */ 
factorial (fact) ; 

/* Output results */ 
for (j =0; j <= 20; 

printf("j = %3d factorial(j) = %12.3e\n", j, fact[j]); 

return 0; 

> 



(n + 1)! = (n + 1)n! 



(2.1) 



int main() 
{ 

int j; 

double fact [21] ; 



// Declaration of array fact[] 




void factorial (double fact[]) 
{ 

/* 

Function to calculate factorials of all integers 
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between and 20 (in form of floating-point 
numbers) via recursion formula 

(n+1) ! = (n+1) n! 

Factorials returned in array fact[0..20] 

*/ 

int count ; 

fact[0] = 1. ; // Set 0! = 1 

/* Calculate 1! through 20! via recursion */ 
for (count = 0; count < 20; ++count) 

f act [count+1] = (double) (count+1) * fact [count]; 

return; 

} 

The output from the above program is identical to that from printf act . c. 

It is important to realize that there is no array bound checking in C. If an array 
x is declared to have 100 elements then the compiler will reserve 100 contigu- 
ous, appropriately sized, slots in computer memory on its behalf. The contents 
of these slots can be accessed via expressions of the form x[i], where the inte- 
ger i should lie in the range to 99. As we have seen, the compiler interprets 
x[i] to mean the contents of the memory slot which is i slots along from the 
beginning of the array. Unfortunately, expressions such as x [100] or x [1000] are 
interpreted in a like manner, leading the compiler to instruct the executable to 
access memory slots which lie off the end of the memory block reserved for x. 
Obviously, accessing elements of an array which do not exist is going to produce 
some sort of error. Exactly what sort of error is very difficult to say — the program 
may crash, it may produce absurdly incorrect output, it may produce plausible 
but incorrect output, it may even produce correct output — it all depends on ex- 
actly what information is being stored in the memory locations surrounding the 
block of memory reserved for x. This type of error can be extremely difficult to 
debug, since it may not be immediately apparent that something has gone wrong 
when the program is executed. It is, therefore, the programmer's responsibility 
to ensure that all references to array elements lie within the declared bounds of 
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the associated arrays. 

Let us now discuss multi-dimensional arrays in more detail. The elements of 
a multi-dimensional array are stored contiguously in a block of computer mem- 
ory. In scanning across this block, from its start to its end, the order of storage is 
such than the last subscript of the array varies most rapidly whilst the first varies 
least rapidly. For instance, the elements of the two-dimensional array x [2] [2] 
are stored in the order: x[0] [0] , x[0] [1] , x[l] [0] , x[l] [1]. The elements 
of a multi-dimensional array can only be addressed if the program is explicitly 
told the size of the array in its second, third, etc. dimensions. It is, therefore, not 
surprising to learn that when a multi-dimensional array is passed to a function, as 
an argument, then the associated argument declaration within the function defi- 
nition must include explicit size declarations in all of the subscript positions except 
the first. The same is true for a multi-dimensional array argument appearing in a 
function prototype. 

2.13 Character Strings 

The basic C character data type is called char. A character string in C can be rep- 
resented symbolically as an array of data type char. For instance, the declaration 

char word [20] = "four"; 

initializes the 20-element character array word, and then stores the character 
string "four" in it. The resulting elements of word are as follows: 

word[0] = >f> word[l] = 'o' word[2] = >u> word[3] = >r> word[4] = '\0' 

with the remaining elements undefined. Here, ' f ' represents the character "f", 
etc., and '\0' represents the so-called null character (ASCII code 0), which is 
used in C to signal the termination of a character string. The null character is 
automatically added to the end of any character string enclosed in double quotes. 
Note that, since all character strings in C must be terminated by the (invisible) 
null character, it takes a character array of size at least n+1 to store an n-letter 
string. 
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As with arrays of numbers, the name of a character array is essentially equiva- 
lent to a pointer to the first element of that array. Hence, word [i] and * (word + i) 
both refer to the same character in the character array word. Note, however, that 
the name of a character array is not a true pointer, since the address to which it 
points cannot be changed. Of course, we can always represent a character array 
using a true pointer. Consider the declaration 

char *word = "four"; 

Here, word is declared to be a pointer to a char which points towards the first 
element of the character string 'f ' 'o' 'u' 'r' '\0\ Unlike the name of a 
character array, a true pointer to a char can be redirected. Thus, 

char *word = "four"; 
word = "five" ; 

is legal, whereas 

char word [20] = "four"; 
word = "five" ; 

is illegal. Note that, in the former example, the addresses of the first elements 
of the strings "four" and "five" are probably different. Of course, the contents 
of a character array can always be changed, element by element — it is just the 
address of the first element which must remain constant. Thus, 

char word [20] = "four"; 

word[0] = 'f>; 

word[l] = 'i>; 

word [2] = 'v'; 

word [3] = 'e'; 

word [4] = '\0>; 

is perfectly legal. 

Note, finally, that a character string can be printed via the print f () function 
by making use of a %s entry in its control string: e.g., 
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printf ("word = %s\n" , word); 

Here, the second argument, word, can either be the name of a character array or 
a true pointer to the first element of a character string. 

2.14 Multi-File Programs 

In a program consisting of many different functions, it is often convenient to place 
each function in an individual file, and then use the make utility to compile each 
file separately and link them together to produce an executable. 

There are a few common-sense rules associated with multi-file programs. Since 
a given file is initially compiled separately from the rest of the program, all sym- 
bolic constants which appear in that file must be defined at its start. Likewise, all 
referenced library functions must be accompanied by the appropriate references 
to header files. Also, any referenced user-defined functions must have their pro- 
totypes at the start of the file. Finally, all global variables used in the file must 
be declared at its start. This usually means that definitions for common sym- 
bolic constants, header files for common library functions, prototypes for com- 
mon user-defined functions, and declarations for common global variables will 
appear in multiple files. Note that a given global variable can only be initialized 
in one of its declaration statements, which is regarded as the true declaration of 
that variable [conventionally, the true declaration appears in the file containing 
the function main()]. Indeed, the other declarations, which we shall term defini- 
tions, must be preceded by the keyword extern to distinguish them from the true 
declaration. 

As an example, let us take the program printf act4. c, listed previously, and 
break it up into multiple files, each containing a single function. The files in 
question are called main.c and factorial, c. The listings of the two files which 
make up the program are as follows: 

/* main.c */ 
/* 

Program to print factorials of all integers 
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between and 20 

*/ 

#include <stdio.h> 

/* Prototype for fucntion factorial () */ 
void factorialQ; 

/* Global variable declarations */ 
int j; 

double fact; 

int main() 
{ 

/* Print factorials of all integers between and 20 */ 
for (j = 0; j <= 20; ++j) 
{ 

factorial () ; 

printf("j = °/,3d factorial (j) = °/„12.3e\n", j, fact); 

} 

return 0; 

} 

and 

/* factorial. c */ 
/* 

Function to evaluate factorial (in floating point form) 
of non-negative integer j. Result stored in variable fact. 

*/ 

#include <stdio.h> 
#include <stdlib.h> 

/* Global variable definitions */ 
extern int j ; 
extern double fact; 

void factorial O 
{ 

int count ; 

/* Abort if j is negative integer */ 
if (j < 0) 
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{ 

printf ("\nError : factorial of negative integer not def ined\n") ; 
exit(l) ; 

} 

/* Calculate factorial */ 

for (count = j, fact = 1.; count > 0; — count) fact *= (double) count; 
return; 

} 

Note that all library functions and user-defined functions referenced in each file 
are declared (either via a header file or a function prototype) at the start of that 
file. Note, also, the distinction between the global variable declarations in the file 
main, c and the global variable definitions in the file factorial . c. 

2.15 Command Line Parameters 

The main() function may optionally possess special arguments which allow pa- 
rameters to be passed to this function from the operating system. There are two 
such arguments, which are conventionally called argc and argv. The former ar- 
gument, argc, is an integer which is set to the number of parameters passed to 
main(), whereas the latter argument, argv, is an array of pointers to character 
strings which contain these parameters. In order to pass one or more parameters 
to a C program when it is executed from the operating system, the parameters 
must follow the program name on the command line: e.g., 

% program-name parameter_l parameter_2 parameter_3 . . . parameter_n 

The program name will be stored in the first item in argv, followed by each of 
the parameters. Hence, if the program name is followed by n parameters there 
will be n + 1 entries in argv, ranging from argv [0] to argv [n] . Furthermore, 
argc will be automatically set equal to n + 1. 

The program listed below is a simple illustration of the use of command line 
parameters: it simply echoes all of the parameters passed to it. 

/* repeat. c */ 
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/* 

Program to read and echo data from command line 

*/ 

int main(int argc, char *argv[]) 
{ 

int i ; 

for (i = 1; i < argc; i++) printf("%s ", argv[i]); 
printf ("\n") ; 

return 0; 

} 

Assuming that the executable is called repeat, the typical output from this pro- 
gram is as follows: 

% repeat The quick brown fox jumped over the lazy hounds 

The quick brown fox jumped over the lazy hounds 

% 



Suppose that one or more of the parameters passed to a given program are 
numbers. As we have seen, these numbers are actually passed as character strings. 
Hence, before they can be employed in calculations, they must be converted into 
either type int or type double. This can be achieved via the use of the functions 
atoi() and atof () (the appropriate header file for these functions is stdlib.h). 
Thus, int atoi(char *ptr) converts a string pointed to by ptr into an int, 
whereas double atof (char *ptr) converts a string pointed to by ptr into an 
double. The program listed below illustrates the use of the atof () function: 
it reads in a number passed as a command line parameter, interprets it as a 
temperature in degrees Fahrenheit, converts it to degrees Celsius, and then prints 
out the result. 

/* ftoc.c */ 

/* 

Program to convert temperature in Fahrenheit input 
on command line to temperature in Celsius 

*/ 

#include <stdlib.h> 
#include <stdio.h> 
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int main(int argc, char *argv[]) 
{ 

double deg_f, deg_c; 

/* If no parameter passed to program print error 

message and exit */ 
if (argc < 2) 
{ 

printf ("Usage : ftoc temperature\n") ; 
exit(l) ; 

} 

/* Convert first command line parameter to double */ 
deg_f = atof (argv [1] ) ; 

/* Convert from Fahrenheit to Celsius */ 
deg_c = (5. / 9.) * (deg_f - 32.); 

printf ("%f degrees Fahrenheit equals %f degrees Celsius\n" , 
deg_f, deg_c) ; 

return 0; 

> 

Assuming that the executable is called ftoc, the typical output from this program 
is as follows: 

% ftoc 98 

98.000000 degrees Fahrenheit equals 36.666667 degrees Celsius 
% 



2.16 Timing 

The header file time . h defines a number of library functions which can be used to 
assess how much CPU time a C program consumes during execution. The simplest 
such function is called clock (). A call to this function, with no argument, will 
return the amount of CPU time used so far by the calling program. The time 
is returned in a special data type, clock_t, defined in time.h. This time must 
be divided by CL0CKS_PER_SEC, also defined in time .h, in order to covert it into 
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seconds. The ability to measure how much CPU time a given code consumes 
is useful in scientific programming: e.g., because it allows the effectiveness of 
the various available compiler optimization flags to be determined. Optimization 
usually (but not always!) speeds up the execution of a program. However, over 
aggressive optimization can often slow a program down again. 

The program listed below illustrates the simple use of the clock () function. 
The program compares the CPU time required to raise a double to the fourth 
power via a direct calculation and via a call to the powO function. Actually 
both operations are performed a million times and the elapsed CPU time is then 
divided by a million. 

/* timing. c */ 
/* 

Program to test operation of clock() function 

*/ 

#include <time.h> 
#include <math.h> 
#define N_LDDP 1000000 

int main() 
{ 

int i ; 

double a = 11234567890123456.0, b; 
clock_t time_l, time_2; 

time_l = clockO ; 

for (i = 0; i < N_L00P; b = a*a*a*a; 

time_2 = clockO ; 

printf ("CPU time needed to evaluate a*a*a*a: °/ f microsecs\n" , 

(double) (time_2 - time_l) / (double) CL0CKS_PER_SEC) ; 

time_l = clockO ; 

for (i = 0; i < N_L00P; b = pow(a, 4.); 

time_2 = clockO ; 

printf ("CPU time needed to evaluate pow(a, 4.): %f microsecs\n" , 
(double) (time_2 - time_l) / (double) CL0CKS_PER_SEC) ; 

return 0; 

} 

The typical output from this program is as follows: 
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CPU time needed to evaluate a*a*a*a: 0.190000 microsecs 
CPU time needed to evaluate pow(a, 4.): 1.150000 microsecs 
% 

Clearly, evaluating a fourth power using the pow() function is a lot more expen- 
sive than the direct calculation. Hence, as has already been mentioned, the pow() 
function should not be used to raise floating point quantities to small integer pow- 
ers. 

2.17 Random Numbers 

A large class of scientific calculations (e.g., so-called Monte Carlo calculations) 
require the use of random variables. A call to the randO function (header file 
stdlib.h), with no arguments, returns a fairly good approximation to a random 
integer in the range to RAND_MAX (defined in stdlib.h). The srandO function 
sets its argument, which is of type int, as the seed for a new sequence of num- 
bers to be returned by randO . These sequences are repeatable by calling srandO 
with the same seed value. If no seed value is provided, the randO function is 
automatically seeded with the value 1. It is common practice in C programming 
to seed the random number generator with the number of seconds elapsed since 
00:00:00 UTC, January 1st, 1970. This number is returned, as an integer, via a 
call to the time (NULL) function (header file: <time.h>). Seeding the genera- 
tor in this manner ensures that a different set of random numbers is generated 
automatically each time the program is run. 

The program listed below illustrates the use of the randO function to construct 
a pseudo-random variable, x, which is uniformly distributed in the range to 1. 
The program calculates 1 7 values of x, and then evaluates the mean and variance 
of these values. 

/* random. c */ 
/* 

Program to test operation of randO function 

*/ 

#include <stdlib.h> 
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#include <stdio.h> 
#include <time.h> 
#define N_MAX 10000000 

int main() 
{ 

int i, seed; 

double sum_0, sum_l, mean, var, x; 

/* Seed random number generator */ 
seed = time (NULL); 
srand(seed) ; 

/* Calculate mean and variance of x: random number uniformly 

distributed in range to 1 */ 
for (i = 1, sum_0 = 0., sum_l =0.; i <= N_MAX; i++) 
{ 

x = (double) rand() / (double) RAND_MAX ; 
sum_0 += x; 

sum_l += (x - 0.5) * (x - 0.5); 

} 

mean = sum_0 / (double) N_MAX; 
var = sum_l / (double) N_MAX; 

printf ("mean(x) = %12.10f var(x) = %12. lOf \n" , mean, var); 
return 0; 

} 

The typical output from this program is as follows: 

mean(x) = 0.5000335261 var(x) = 0.0833193874 
% 

As is easily demonstrated, the theoretical mean and variance of x are 1 /2 and 
1 /1 2, respectively. It can be seen that the values returned by the program agree 
with these theoretical values to five decimal places, which is all that can be ex- 
pected with only 10 7 calls. 
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2.18 C++ Extensions to C 

In this subsection, we shall briefly discuss some of the useful, non-object-orientated 
extensions to C introduced in the C+ + language. Files containing source code 
which incorporates C+ + elements should be distinguished from files containing 
plain C code via the extension . cpp. 

In C, all local variables within a function must be declared before the first 
executable statement. In C + + , this restriction is relaxed: a local variable can be 
declared (almost) anywhere in a function, provided that this declaration occurs 
prior to the variable's first use. The following code snippet illustrates the use of 
this new feature: 

for (int i = 0; i < MAX; i++) 
{ 

> 

Observe that the index i of the for loop is now declared at the start of the 
loop, instead of at the start of the function containing the loop. This is far more 
convenient, and also makes the code easier to read (since we no longer have to 
skip back to the declarations at the start of the function to check that i is an 
int). Note, however, that the variable i is only defined over the extent of the 
loop (i.e., between the curly braces) . Any attempt to reference i outside the loop 
will result in a compilation error. In general, when a variable is declared in C + + 
its scope (i.e., range of definition) extends from its declaration to the closing curly 
brace which terminates the current program block. Program blocks are functions, 
loops, conditionally executed compound statements, etc., and are delineated by 
curly braces. There are a number of restrictions to this new method of variable 
declaration. Variables cannot be declared within conditional statements, in the 
second or third expressions of for loops, or in function calls. 

In C, we have seen that in order to pass an argument to a function in such 
a manner that changes made to this argument within the function are passed 
back to the calling routine, we must actually pass a pointer to the argument. This 
procedure, which is called passing by reference, is illustrated in the code snippet 
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listed below: 



void square (double , double *) ; 

int main() 
{ 

double arg, res; 
square(arg, feres); 

return 0; 

> 

void square (double x, double *y) 
{ 

*y = x * x; 
return; 

> 

Here, the second argument to square () is returned to mainO in modified form. 
This argument must, therefore, be passed as a pointer. Consequently, we must 
write feres, rather that res, when calling square (), and we must refer to the 
argument as *y, rather than y, in the function itself. After a while, all these 
ampersands and asterisks can become a little tedious! C++ introduces a new 
method of passing by reference which somewhat less involved. This new method 
is illustrated in the following: 



void square (double , double &) ; 

int main() 
{ 

double arg, res; 
square(arg, res); 

return 0; 

> 

void square (double x, double fey) 
{ 

y = x * x; 
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return; 

} 

Here, the second argument to square () is again passed by reference. However, 
this is now indicated by prepending an ampersand to the variable name in the 
function declaration. A corresponding ampersand appears in the associated func- 
tion prototype. Note that we do not need to explicitly pass a pointer to the second 
argument when calling square () (this is done behind the scenes): i.e., we write 
res, rather than feres, when calling square (). Likewise, we do not have to ex- 
plicitly deference the argument in the function itself (this is also done behind the 
scenes): i.e., we refer to the argument by its regular local name, y, as opposed to 
*y, within the function. 

Functions are used in C programs to avoid having to repeat the same block 
of code in many different places in the source. The use of functions also ren- 
ders a code easier to read and maintain. However, there is a price to pay for 
the convenience of functions. When a regular function is called in an executable, 
the program jumps to the block of memory in which the compiled function code 
is stored, and then jumps back to its original position in memory space when 
the function returns. Unfortunately, the large jumps in memory space associated 
with a function call take up a non-negligible amount of CPU time. Indeed, the 
overhead associated with making function calls often discourages scientific pro- 
grammers from writing small functions, even when it may be desirable to do so. 
C+ + provides a way out of this dilemma, via the use of the new keyword inline. 
An inline function looks like a normal function when it is used, but is compiled in 
a different manner. Calling an inline function from several different locations in 
a code does not result in multiple calls to a single function. Instead, the code for 
the inline function is inserted into the program code by the compiler wherever 
the function is used. 

Inline functions are only useful for small functions. The disadvantage of in- 
serting the code for a large function multiple times into the code for a typical 
program easily outweighs the small gain in performance obtained by the elim- 
ination of standard function calls. The break-even point for inline functions is 
usually about three executable lines. 
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To inline a function, a programmer adds the keyword inline at the start of 
the function's definition. For example: 

inline double square (double x) 
{ 

return x * x; 

} 

Because the body of an inline function must be known before the compiler can 
insert it into the program code, wherever the function is used, we must define 
such a function prior to its first use — a prototype declaration is not enough. It is 
common practice to define inline functions at the same location in source code 
files that the prototypes for regular (i.e., outline) functions are placed. 

Variable size array declarations of the form 

void function (n) 
{ 

int n; 

double x [n] ; 

} 

are illegal in C, which is extremely inconvenient. In C++, such declarations 
are enabled via the use of the new keywords new and delete. Thus, the C + + 
implementation of the above code snippet takes the form: 

void function (n) 
{ 

int n; 

double *x = new double [n] ; 
x[i] = . . . 



delete x [] ; 

> 
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Note that x is actually declared as a pointer, rather than a standard array. The 
declaration new double [n] reserves a memory block which is just large enough 
to store n doubles, and then returns the address of the start of this block. The line 
delete x[] frees up the block of memory associated with the array x when it is 
no longer needed (note that this is not done automatically). Unfortunately the 
new and delete keywords cannot be used to make variable size multi-dimensional 
arrays. 



2.19 Complex Numbers 

As we have already mentioned, the C language definition does not include com- 
plex arithmetic — presumably because the square root of minus one is not a con- 
cept which crops up very often in systems programming! Fortunately, this rather 
serious deficiency — at least, as far as the scientific programmer is concerned — is 
remedied in C + + . The program listed below illustrates the use of the C+ + com- 
plex class (header file complex.h) to perform complex arithmetic using doubles: 

/* complex. cpp */ 
/* 

Program to test out C++ complex class 

*/ 

#include <complex.h> 
#include <stdio.h> 

/* Define complex double data type */ 
typedef complex<double> dcomp; 

int main() 
{ 

dcomp i, a, b, c, d, e, p, q, r; // Declare complex double variables 
double x, y; 

/* Set complex double variable equal to complex double constant */ 
i = dcomp (0 . , 1 . ) ; 

printf("\ni = (°/.6.4f, °/,6.4f)\n", i) ; 

/* Test arithmetic operations with complex double variables */ 
a = i * i; 
b = 1. / i; 
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printf ("\ni*i = (°/„6.4f, %6.4f)\n\ a); 
printfO'l/i = (°/.6.4f, °/,6.4f )\n" , b) ; 

/* Test mathematical functions using complex double variables */ 

c = sqrt (i) ; 

d = sin(i) ; 

e = pow(i , . 25) ; 

printf ("\nsqrt(i) = C/.6.4f, %6.4f)\n", c) ; 
printf ("sin (i) = (%6.4f, %6.4f)\n\ d) ; 
printf ("i~0. 25 = (%6.4f, %6.4f)\n", e) ; 

/* Test complex operations */ 
p = conj (i) ; 
q = real (i) ; 
r = imag(i) ; 

printf ("\nconj(i) = (°/„6.4f, %6.4f)\n", p) ; 
printf ("real (i) = %6.4f\n", q) ; 
printf ("imag(i) = %6.4f\n\ r) ; 

return 0; 

} 

The typical output from this program is as follows: 

i = (0.0000, 1.0000) 

i*i = (-1.0000, 0.0000) 
1/i = (0.0000, -1.0000) 

sqrt(i) = (0.7071, 0.7071) 
sin(i) = (0.0000, 1.1752) 
i'0.25 = (0.9239, 0.3827) 

conj(i) = (0.0000, -1.0000) 
real(i) = 0.0000 
imag(i) = 1.0000 
% 

The program first of all defines the complex double type dcomp. Variables of this 
type are then declared, set equal to complex constants, employed in arithmetic 
expressions, used as the arguments of mathematical functions, etc., in much the 
same manner that we would perform similar operations in C with variables of 
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type double. Note the special functions conj (), real(), and imagO, which take 
the complex conjugate of, find the real part of, and find the imaginary part of a 
complex variable, respectively. 

2.20 Variable Size Multi-Dimensional Arrays 

Multi-dimensional arrays crop up in a wide variety of different applications in 
scientific programming. Moreover, it is very common for the sizes of the arrays 
employed in a scientific code to vary from run to run, depending on the partic- 
ular values of the input parameters. For instance, in a standard fluid code the 
array sizes depend on the number of grid points, which, in turn, depends on 
the requested accuracy. Thus, a crucial test of the suitability of a programming 
language for scientific purposes is its ability to deal with variable size, multi- 
dimensional arrays in a convenient manner. Unfortunately, C fails this test rather 
badly, since variable size matrix declarations of the form 

void function(a, m, n) 
{ 

int m, n; 
double a[m] [n] ; 

are illegal (unlike in FORTRAN 77, where variable size matrix declarations are 
perfectly acceptable). Indeed, this unfortunate (and quite unnecessary!) omis- 
sion in the C language definition is often put forward as a reason why C should 
not be used for scientific purposes. Fortunately, we can get around the absence 
of variable size multi-dimensional arrays in C by making use of a freely available 
C+ + package called the Blitz + + library — see http : //www . oonumerics . org/blitz/. 

The program listed below illustrates the use of the Blitz ++ library. The pro- 
gram adds together two matrices whose dimensions and elements are input by 
the user, and then prints out the result. 

/* addmatrix.c */ 
/* 

Program to add two variable dimension matrices input by user 

*/ 
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#include <stdio.h> 
#include <stdlib.h> 
#include <blitz/array .h> 

using namespace blitz; 

/* Function prototypes */ 
void readin(Array<double,2>) ; 
void writeout (Array<double , 2>) ; 

void addmatrices (Array<double , 2> , Array<double,2>, Array<double , 2>) ; 

int main() 
{ 

int n, m; 

/* Input number of rows and columns */ 

printf ("\nPlease input number of rows, n, and number of columns, m: "); 
scanf ('7,d %d\ fen, ton) ; 

/* Check that n, m are positive integers */ 
if (n <= | | m <= 0) 
{ 

printf ("\nError: invalid values for n and/or m\n"); 
exit(l) ; 

} 

/* Array declarations */ 

Array<double , 2> A(n, m) , B(n, m) , C(n, m) ; 

/* Read in elements of A, row by row */ 
printf ("\nReading in elements of A:\n"); 
readin(A) ; 

/* Read in elements of B, row by row */ 
printf ("\nReading in elements of B:\n"); 
readin(B) ; 

/* Write out elements of A, row by row */ 
printf ("\nWriting out elements of A:\n"); 
writeout (A) ; 

/* Write out elements of B, row by row */ 
printf ("\n\nWriting out elements of B:\n"); 
writeout (B) ; 
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/* Add matrices A and B */ 
addmatrices (A, B, C) ; 

/* Write out matrix C = A + B, row by row */ 
printf ("\n\nWriting out elements of C = A + B:\n"); 
writeout (C) ; 
printf ("\n") ; 

return 0; 

} 

/ /o/ <yo/ o/o/o/ o/o/ °/ / / °/°/ °/°/ o / / / / °/°/ °/°/ °/°/ v v v v vyy v v v v v / / / / / / / / / / / / / / / / / / / / / / 

/ / /o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o A 

/* 

Read in elements of matrix M, row by row 

*/ 

void readin(Array<double,2> M) 
{ 

int n = M. extent (0); 
int m = M. extent (1); 

for (int i = 0; i < n; i++) 
{ 

printf ("\nRow °/„d: ", i + 1) ; 

for (int j = 0; j < m; j++) scanf ("°/„lf " , &M(i, j)); 

} 

return; 

} 

/ /0/0/ 0/0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/ 0/0/ 0/0/ 0/0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/ 0/0/ 0/0/ 0/0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/0/ 

/ / /o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o /o/o 

/* 

Write out elements of matrix M, row by row 

*/ 

void writeout (Array<double , 2> M) 
{ 

int n = M. extent (0); 
int m = M. extent (1); 

for (int i = 0; i < n; i++) 
{ 

printf ("\nRow °/„d: ", i + 1) ; 
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for (int j = 0; j < m; j++) printf ( ,,0 /„7 . 2f 11 , M(i, j)); 

} 

return ; 

} 

/ 70/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/ 0/0/0/ 0/0/ 0/0/ 0/0/ 

/ / /o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/o/oA 

/* 

Add matrices M and N and store result in matrix P 

*/ 

void addmatrices (Array<double , 2> M, Array<double,2> N, Array<double,2> P) 
{ 

int n = M. extent (0) ; 
int m = M.extent(l); 

for (int i = 0; i < n; i++) 
for (int j =0; j < m; 

P(i, j) = M(i, j) + N(i, j); 

return; 

} 

Typical output from the program looks like 



Please input number of rows, n, and number of columns, m: 3 3 
Reading in elements of A: 
Row 1: 14 5 
Row 2:678 
Row 3: 3 6 

Reading in elements of B: 
Row 1:16 
Row 2:478 
Row 3: 4 8 8 
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Writing out elements of A: 

Row 1: 1.00 4.00 5.00 
Row 2: 6.00 7.00 8.00 
Row 3: 3.00 6.00 0.00 

Writing out elements of B: 

Row 1: 1.00 6.00 0.00 
Row 2: 4.00 7.00 8.00 
Row 3: 4.00 8.00 8.00 

Writing out elements of C = A + B: 

Row 1: 2.00 10.00 5.00 
Row 2: 10.00 14.00 16.00 
Row 3: 7.00 14.00 8.00 

The header file for the Blitz++ library is called blitz/array .h. Moreover, any 
program file which makes use of Blitz ++ must include the cryptic line using 
namespace blitz; before the first call, otherwise compilation errors will ensue. 
The declaration Array<double , 2> A(n, m) declares a two-dimensional n by m 
array of doubles. The generalization to an array of integers, or a higher dimen- 
sion array, is fairly obvious. Note that the i, j element of matrix A is refered to 
simply as A ( i , j ) . The function call M . extent (0) returns the size of array M in its 
first dimension. Likewise, M. extent (1) returns the size of array M in its second 
dimension, etc. More details of the operation of Blitz + + can be found by reading 
the extensive documentation which accompanies this library. Unfortunately, the 
Blitz + + library slows down compilation considerably since it makes use of some 
very advanced templating features of the C++ language. 

2.21 The CAM Graphics Class 

There are a myriad of useful, prewritten C+ + classes which are freely available 
on the web — for more details see http : //www. trumphurst . com/cpplibsx.html. 
In this Subsection, we shall discuss just one of these — namely, the CAM graphics 
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class whose purpose is to enable a C+ + program to generate simple line plots. 
This class is freely available from the following URL: http : //www . math . ucla . edu/- 
"anderson/CAMclass/CAMClass . html 



The GAM graphics class actually generates a PostScripts file. Postscript is a 
programming language that describes the appearance of a printed page. It was 
developed by Adobe in 1985, and has become an industry standard for printing 
and imaging. All major printer manufacturers make printers that can interpret 
PostScript. A PostScript file is conventionally identified via a .ps suffix. The 
schematic code listed below illustrates the basic use of the GAM graphics class: 



#include <gprocess .h> // Header file for CAM graphic class 



CAMgraphicsProcess Gprocess; // declare a graphics process 

CAMpostScriptDriver Pdriver("f ilename.ps") ; // declare a PostScript driver 
Gprocess . attachDriver (Pdr iver) ; // attach driver to process 



Gprocess .frameO ; 
Gprocess .frameO ; 



// "frame" the first plot 
// "frame" the second plot 



Gprocess .frameO ; 
Gprocess . detachDriver () ; 



// "frame" the last plot 
// detach the driver 



The header file for the class is called gprocess. h. The procedure for generating 
a plot is to first declare a graphics process, then declare a PostScript driver and 
attach this to a PostScript file — f ilename .ps, in the above example — and, finally, 
attach this driver to the process. A PostScript file can contain multiple pictures, 
or frames. Each frame is terminated by a call to Gprocess. frameO. Finally, the 
driver is detached, which has the effect of closing the PostScript file. 

The program listed below uses the CAM graphics class to plot the curve y = 
sin x for x in the range — In to 2n. 

/* camgraphl . cpp */ 

12 The CAM graphics class is copyrighted to its author, Prof. Chris Anderson, Department of Mathematics, UCLA, 
1998. 

"PostScript is a registered trademark of Adobe Systems Incorporated. 
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/* 

Illustration of use of CAM graphics class to create simple line plot 
Program plots y = sin~2 x versus x in range -2 PI to +2 PI 
Program adapted from gpsmpl.cpp by Chris Anderson, UCLA 1996 

*/ 

#include <gprocess.h> 
#include <math.h> 
#include <stdlib.h> 

double func (double) ; 

int main() 
{ 

int N_points = 400; 

double x_start = -2. * M_PI; 

double x_end = 2. * M_PI; 

double delta_x = (x_end - x_start) / ((double) N_points - 1.); 

double *x = new double [N_points] ; 
double *y = new double [N_points] ; 

for (int i = 0; i < N_points; i++) 
{ 

x[i] = x_start + (double) i * delta_x; 
y [i] = func(x[i] ) ; 
x[i] /= M_PI; 

} 

{ // This brace used to limit scope of Gprocess 

CAMgraphicsProcess Gprocess; // declare a graphics process 

CAMpostScriptDriver PdriverCgraphl.ps"); // declare a PostScript driver 
Gprocess . attachDriver (Pdriver) ; // attach driver to process 

Gprocess . setAxisRange (-2 . , 2., -2., 2.); // set plotting ranges 

Gprocess. title("y = sin(x*x)"); // label the plot 
Gprocess. labelXC'x / PI"); 
Gprocess. labelYC'y") ; 

Gprocess. plot(x, y, N_points) ; // do the plotting 

Gprocess . frame () ; // "frame" the plot 
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Gprocess.detachDriverO ; // detach the driver 

} // This brace calls the destructor for Gprocess: 
// without it the system () call would hang up 

delete [] x; 
delete [] y; 

systemC'gv graphl.ps"); // display plot on screen 
return 0; 

> 

double func (double x) 
{ 

return sin(x*x) ; 

> 

The command Gprocess .plot (x, y, n) plots the n values of vector y against 
the n values of vector x as a solid curve. The command Gprocess . setAxisRange- 
(x_low, x_high, y_low, y_high) sets the range of plotting. Finally, the com- 
mands Gprocess .title ("title"), Gprocess . labelX("x_label"), and Gprocess . - 
labelY("y_label") label the plot, the x-axis, and the u-axis, respectively. Inci- 
dentally, the UNIX function call systemC'gv graphl.ps") is used to pass the 
command gv graphl.ps to the operating system. On execution, this command 
displays the contents of graphl .ps on the screen. The graph written in the file 
graphl .ps is shown in Fig [I] 

The program shown below illustrates some of the more advanced features of 
the CAM graphics class: 

/* camgraph2 . cpp */ 
/* 

Illustration of use of CAMgraphics class to create more advanced line plots 

Program plots three trigonometric functions versus x in range 
-2 PI to +2 PI using different plot styles and different 
line styles 

Program adapted from gpsmp2.cpp by Chris Anderson, UCLA 1996 

*/ 
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y = sin(x*x) 

2.00 1 , 1 , 1 r 




Figure 1: An example plot generated by the CAM graphics class. 



#include <gprocess .h> 
#include <math.h> 
#include <stdlib.h> 



double funl (double) 
double fun2 (double) 
double fun3 (double) 



int main() 
{ 

int N_points = 100; 

double x_start = -2. * M_PI; 

double x_end = 2. * M_PI; 

double delta_x = (x_end - x_start) / ((double) N_points 



- l.); 



double *x = new double [N_points] 

double *yl = new double [N_points] 

double *y2 = new double [N_points] 

double *y3 = new double [N_points] 



for (int i = 0; i < N_points; i++) 
{ 

x[i] = x_start + (double) i * delta_x; 
yl[i] = funl(x[i]) 
y2[i] = fun2(x[i]) 
y3[i] = fun3(x[i]) 
x[i] /= M_PI; 
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{ 

CAMgraphicsProcess Gprocess; // declare a graphics process 

CAMpostScriptDriver Pdriver(" graph2.ps") ; // declare a PostScript driver 

Gprocess . attachDriver (Pdriver) ; // attach driver to process 

/* First frame; using different plot "styles" */ 

Gprocess . setAxisRange (-2 . , 2., -2., 2.); // set plotting ranges 

Gprocess. title("Plots Using Different Plot Styles");// label the plot 
Gprocess. labelXC'x / PI"); 
Gprocess. labelYC'y") ; 

Gprocess. plot (x, yl, N_points) ; // solid line (default) 

Gprocess. plot (x, y2, N_points, '+'); // + markers 

Gprocess. plot(x, y3, N_points, '+' , 2); // + markers and solid line 

Gprocess . frame () ; // "frame" the plot 

/* Second frame; using different plot line "styles" */ 

Gprocess . setAxisRange (-2 . , 2., -2., 2.); // set plotting ranges 

Gprocess. title("Plots Using Different Line Styles");// label the plot 
Gprocess. labelXC'x / PI"); 
Gprocess. labelYC'y") ; 

Gprocess. plot(x, yl, N_points) 
Gprocess . setPlotDashPattern(l) 
Gprocess. plot (x, y2, N_points) 
Gprocess . setPlotDashPattern(4) 
Gprocess .plot (x, y3, N_points) 

Gprocess . f rame () ; 

Gprocess. detachDriver () ; 

} 

delete [] x; 
delete [] yl; 
delete [] y2; 
delete [] y3; 

system("gv graph2.ps"); // display plots on screen 
return 0; 



// solid line (default) 
// dashed line 
// dashed-dot line 
// "frame" the plot 
// detach the driver 
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double f unl (double x) 
{ 

return sin(x) ; 

} 

double fun2 (double x) 
{ 

return cos (x) ; 

} 

double fun3 (double x) 
{ 

return cos(2.*x); 

> 



The command Gprocess . plot (x , y , n , ' + ' ) plots the n values of vector y against 
the n values of vector x as a set of points, each indicated by a ' + ' character. The 
command Gprocess. plot (x, y, n, ' + ' , 2) does the same, but also connects 
the points with a solid line. The fourth argument of this command is an inte- 
ger code which determines the plot style. The various options are as follows: 
- curve; 1 - points; 2 - curve and points. The command Gprocess . setPlot- 
DashPattern(n) sets the line style. The argument is again an integer code. The 
various options are: - solid; 1 - dash; 2 - double-dash; 4 - dash-dot; 5 - dash- 
double-dot; 6 - dots. 

The graphs written in the first and second frames of graph2.ps are shown in 
Figs. El and [31 respectively. 
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Figure 2: An example plot generated by the CAM graphics class. 
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Figure 3: An example plot generated by the CAM graphics class. 



100 



3 INTEGRATION OF ODES 



3 Integration of ODEs 



3.1 Introduction 



In this section, we shall discuss the standard numerical techniques used to inte- 
grate systems of ordinary differential equations (ODEs). We shall then employ 
these techniques to simulate the trajectories of various different types of baseball 
pitch. 

By definition, an ordinary differential equation, or o.d.e., is a differential equa- 
tion in which all dependent variables are functions of a single independent vari- 
able. Furthermore, an nth-order o.d.e. is such that, when it is reduced to its 
simplest form, the highest order derivative it contains is nth-order. 

According to Newton's laws of motion, the motion of any collection of rigid 
objects can be reduced to a set of second-order o.d.e.s. in which time, t, is the 
common independent variable. For instance, the equations of motion of a set of 
n interacting point objects moving in 1 -dimension might take the form: 



d 2 xj Fj(xi,...,x n ,t) 



dt 2 m.j 



(3.1) 



for j = 1 to n, where xj is the position of the jth object, rrij is its mass, etc. 
Note that a set of n second-order o.d.e.s can always be rewritten as a set of 2n 
first-order o.d.e.s. Thus, the above equations of motion can be rewritten: 



dxj 

dt = V " 
dt m.j 



= v if (3.2) 

(3.3) 



for j = 1 to n. We conclude that a general knowledge of how to numerically solve 
a set of coupled first-order o.d.e.s would enable us to investigate the behaviour 
of a wide variety of interesting dynamical systems. 
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3.2 Euler's Method 



Consider the general first-order o.d.e., 

y' = f(x,u), (3-4) 

where ' denotes d/dx, subject to the general initial-value boundary condition 

vi^o)=Vo- (3.5) 

Clearly, if we can find a method for numerically solving this problem, then we 
should have little difficulty generalizing it to deal with a system of n simultaneous 
first-order o.d.e.s. 

It is important to appreciate that the numerical solution to a differential equa- 
tion is only an approximation to the actual solution. The actual solution, y[x), 
to Eq. (13. 4D is (presumably) a continuous function of a continuous variable, x. 
However, when we solve this equation numerically, the best that we can do is to 
evaluate approximations to the function y[x) at a series of discrete grid-points, 
the x n (say), where n = 0, 1 , 2, • • • and xo < xi < Xi • • •. For the moment, we shall 
restrict our discussion to equally spaced grid-points, where 



xo + uh. (3.6) 



Here, the quantity h is referred to as the step-length. Let y n be our approximation 
to y[x) at the grid-point x n . A numerical integration scheme is essentially a 
method which somehow employs the information contained in the original o.d.e., 
Eq. (13. 4D , to construct a series of rules interrelating the various y n . 



The simplest possible integration scheme was invented by the celebrated 18th 
century Swiss mathematician Leonhard Euler, and is, therefore, called Euler's 
method. Incidentally, it is interesting to note that virtually all of the standard 
methods used in numerical analysis were invented before the advent of electronic 
computers. In olden days, people actually performed numerical calculations by 
hand — and a very long and tedious process it must have been! Suppose that we 
have evaluated an approximation, y n , to the solution, y[x), of Eq. (13. 4D at the 
grid-point x n . The approximate gradient of y[x) at this point is, therefore, given 
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y(x) 



X 



x r 



X, 



"0 "1 2 

Figure 4: Illustration ofEuler's method. 



by 

y' n = f(x n ,y n ). (3.7) 

Let us approximate the curve y (x) as a straight-line between the neighbouring 
grid-points x n and x n+ i . It follows that 



Un+i = y n + y' n K 



(3.8) 



or 



Un+1 = Vn + f (Xu, Vn) H. (3.9) 

The above formula is the essence of Euler's method. It enables us to calculate all 
of the y n , given the initial value, yo, at the first grid-point, xo- Euler's method is 
illustrated in Fig. @] 



3.3 Numerical Errors 



There are two major sources of error associated with a numerical integration 
scheme for o.d.e.s: namely, truncation error and round-off error : Truncation error 
arises in Euler's method because the curve y[x) is not generally a straight-line 
between the neighbouring grid-points x n and x n+ i, as assumed above. The error 
associated with this approximation can easily be assessed by Taylor expanding 
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y(x) about x = x n : 



u(x n + h) = y(x n ) + hy'{x n ) + — y"{x n ) H 

h 2 

= u n + Hf(x n ,-y n ) + — y (xn) H . 



(3.10) 



A comparison of Eqs. (I3.9D and (13.1 0D yields 



"yn+i =Un + Hf(x n ,-y n ) + 0(h 2 ). 



(3.11) 



In other words, every time we take a step using Euler's method we incur a trun- 
cation error of 0(h 2 ), where h is the step-length. Suppose that we use Euler's 
method to integrate our o.d.e. over an x-interval of order unity. This requires 
0(h ] ) steps. If each step incurs an error of 0(h 2 ), and the errors are simply 
cumulative (a fairly conservative assumption), then the net truncation error is 
0(h). In other words, the error associated with integrating an o.d.e. over a finite 
interval using Euler's method is directly proportional to the step-length. Thus, if 
we want to keep the relative error in the integration below about 1 6 then we 
would need to take about one million steps per unit interval in x. Incidentally, 
Euler's method is termed a first-order integration method because the truncation 
error associated with integrating over a finite interval scales like h 1 . More gen- 
erally, an integration method is conventionally called nth order if its truncation 
error per step is 0(h n+1 ). 

Note that truncation error would be incurred even if computers performed 
floating-point arithmetic operations to infinite accuracy. Unfortunately, comput- 
ers do not perform such operations to infinite accuracy. In fact, a computer is only 
capable of storing a floating-point number to a fixed number of decimal places. 
For every type of computer, there is a characteristic number, r\, which is defined as 
the smallest number which when added to a number of order unity gives rise to 
a new number: i.e., a number which when taken away from the original number 
yields a non-zero result. Every floating-point operation incurs a round-off error 
of O(rj) which arises from the finite accuracy to which floating-point numbers 
are stored by the computer. Suppose that we use Euler's method to integrate 
our o.d.e. over an x-interval of order unity. This entails 0(h. _1 ) integration steps, 
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and, therefore, 0(H ] ) floating-point operations. If each floating-point opera- 
tion incurs an error of O(n), and the errors are simply cumulative, then the net 
round-off error is (n/H) . 

The total error, e, associated with integrating our o.d.e. over an x-interval of 
order unity is (approximately) the sum of the truncation and round-off errors. 
Thus, for Euler's method, 

e-^ + h. (3.12) 
h 

Clearly, at large step-lengths the error is dominated by truncation error, whereas 
round-off error dominates at small step-lengths. The net error attains its mini- 
mum value, eo ~ rj 1/2 , when h = ho ~ n 1/2 . There is clearly no point in making the 
step-length, H, any smaller than ho, since this increases the number of floating- 
point operations but does not lead to an increase in the overall accuracy. It is 
also clear that the ultimate accuracy of Euler's method (or any other integration 
method) is determined by the accuracy, n, to which floating-point numbers are 
stored on the computer performing the calculation. 

The value of r| depends on how many bytes the computer hardware uses to 
store floating-point numbers. For IBM-PC clones, the appropriate value for double 
precision floating point numbers is r\ = 2.22 x 10 16 (this value is specified in 
the system header file float. h). It follows that the minimum practical step- 
length for Euler's method on such a computer is ho ~ 1 8 , yielding a minimum 
relative integration error of eo ~ 10 8 . This level of accuracy is perfectly adequate 
for most scientific calculations. Note, however, that the corresponding rj value 
for single precision floating-point numbers is only rj = 1.19 x 10 7 , yielding a 
minimum practical step-length and a minimum relative error for Euler's method 
of ho ~ 3 x 1 4 and eo ~ 3 x 1 4 , respectively. This level of accuracy is generally 
not adequate for scientific calculations, which explains why such calculations are 
invariably performed using double, rather than single, precision floating-point 
numbers on IBM-PC clones (and most other types of computer) . 
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3.4 Numerical Instabilities 



Consider the following example. Suppose that our o.d.e. is 



y' = -ocy 



(3.13) 



where <x> 0, subject to the boundary condition 



y(0) = i. 



(3.14) 



Of course, we can solve this problem analytically to give 



y{x) = exp(— ocx). 



(3.15) 



Note that the solution is a monotonically decreasing function of x. We can also 
solve this problem numerically using Euler's method. Appropriate grid-points are 



Note one curious fact. If H > 2/oc then |u n +il > \yn\- In other words, if the 
step-length is made too large then the numerical solution becomes an oscillatory 
function of x of monotonically increasing amplitude: i.e., the numerical solution 
diverges from the actual solution. This type of catastrophic failure of a numer- 
ical integration scheme is called a numerical instability. All simple integration 
schemes become unstable if the step-length is made sufficiently large. 

3.5 Runge-Kutta Methods 

There are two main reasons why Euler's method is not generally used in scientific 
computing. Firstly, the truncation error per step associated with this method is 
far larger than those associated with other, more advanced, methods (for a given 
value of h). Secondly, Euler's method is too prone to numerical instabilities. 



x n = n h, 



(3.16) 



where n = 0,1,2,---. Euler's method yields 



-y n+1 = (1 - oth)y n . 



(3.17) 
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The methods most commonly employed by scientists to integrate o.d.e.s were 
first developed by the German mathematicians C.D.T. Runge and M.W. Kutta in 



the latter half of the nineteenth centuryo The basic reasoning behind so-called 
Runge-Kutta methods is outlined in the following. 

The main reason that Euler's method has such a large truncation error per 
step is that in evolving the solution from x n to x n+ i the method only evaluates 
derivatives at the beginning of the interval: i.e., at x n . The method is, therefore, 
very asymmetric with respect to the beginning and the end of the interval. We 
can construct a more symmetric integration method by making an Euler-like trial 
step to the midpoint of the interval, and then using the values of both x and y at 
the midpoint to make the real step across the interval. To be more exact, 

k] = hf(x n ,-y n ), (3.18) 
k 2 = hffo + 11/2,^ + 1^/2), (3.19) 
Un+i = u n + k 2 + 0(h 3 ). (3.20) 

As indicated in the error term, this symmetrization cancels out the first-order 
error, making the method second-order. In fact, the above method is generally 
known as a second-order Runge-Kutta method. Euler's method can be thought of 
as a first-order Runge-Kutta method. 

Of course, there is no need to stop at a second-order method. By using two 
trial steps per interval, it is possible to cancel out both the first and second-order 
error terms, and, thereby, construct a third-order Runge-Kutta method. Likewise, 
three trial steps per interval yield a fourth-order method, and so on£H 

The general expression for the total error, e, associated with integrating our 
o.d.e. over an x-interval of order unity using an nth-order Runge-Kutta method 
is approximately 

e^ + H n . (3.21) 
Here, the first term corresponds to round-off error, whereas the second term 



u Numerical recipes in C: the art of scientific computing, W.H. Press, S.A. Teukolsky, W.T. Vettering, and B.R. Flannery 
(Cambridge University Press, Cambridge UK, 1992), p. 710. 
ls Handbook of mathematical functions, M. Abramowitz and LA. Stegun (Dover, New York NY, 1965), p. 896. 
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n 


h 


e 


1 


1.5 x 10 


-8 


1.5 x 10 


-8 


2 


6.1 x 10 


-6 


3.7 x 10- 


-11 


3 


1.2 x 10 


-4 


1.8 x 10 


-12 


4 


7.4 x 10 


-4 


3.0 x 10 


-13 


5 


2.4 x 10 


-3 


9.0 x 10- 


-14 



Table 1: The minimum practical step-length, h , and minimum error, e , for an nth-order Runge- 
Kutta method integrating over a finite interval using double precision arithmetic on an IBM-PC clone. 



represents truncation error. The minimum practical step-length, Ho, and the min- 
imum error, eo, take the values 

H - u 1/(n+1) , (3-22) 
eo - rj n/(n+1) , (3.23) 

respectively. In Tab.CQ, these values are tabulated against n using r\ = 2.22 x 10 16 
(the value appropriate to double precision arithmetic on IBM-PC clones). It can 
be seen that Ho increases and eo decreases as n gets larger. However, the relative 
change in these quantities becomes progressively less dramatic as n increases. 

In the majority of cases, the limiting factor when numerically integrating an 
o.d.e. is not round-off error, but rather the computational effort involved in cal- 
culating the function f(x,u). Note that, in general, an nth-order Runge-Kutta 
method requires n evaluations of this function per step. It can easily be appreci- 
ated that as n is increased a point is quickly reached beyond which any benefits 
associated with the increased accuracy of a higher order method are more than 
offset by the computational "cost" involved in the necessary additional evaluation 
of f (x,u) per step. Although there is no hard and fast general rule, in most prob- 
lems encountered in computational physics this point corresponds to n = 4. In 
other words, in most situations of interest a fourth-order Range Kutta integration 
method represents an appropriate compromise between the competing require- 
ments of a low truncation error per step and a low computational cost per step. 

The standard fourth-order Runge-Kutta method takes the form: 

k! = Hf(x n ,-yJ, (3.24) 
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k 3 



k4 



k 2 



hf(x n + h/2,-y n + k 1 /2), 
hf(x n + h/2,-y n + k 2 /2), 
hf(x n + h,y n + k 3 ), 



(3.25) 
(3.26) 



(3.27) 



Un+1 




(3.28) 



This is the method which we shall use, throughout this course, to integrate first- 
order o.d.e.s. The generalization of this method to deal with systems of coupled 
first-order o.d.e.s is (hopefully) fairly obvious. 

3.6 An Example Fixed- Step RK4 routine 

Listed below is an example fixed-step, fourth-order Run ge-Ku tta (RK4) integra- 
tion routine which utilizes the Blitz++ library (see Sect. 12.201) . 

// rk4_f ixed. cpp 
/* 

Function to advance set of coupled first-order o.d.e.s by single step 
using fixed step-length fourth-order Runge-Kutta scheme 

x ... independent variable 

y ... array of dependent variables 

h ... fixed step-length 

Requires right-hand side routine 

void rhs_eval (double x, Array<double , 1> y, Array<double , 1>& dydx) 

which evaluates derivatives of y (w.r.t. x) in array dydx 



#include <blitz/array .h> 
using namespace blitz; 

void rk4_fixed (doublefe x, Array<double , 1>& y, 

void (*rhs_eval) (double , Array<double , 1> , Array<double , 1>&) , 
double h) 



*/ 
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// Array y assumed to be of extent n, where n is no. of coupled 

// equations 

int n = y. extent (0); 

// Declare local arrays 

Array<double, 1> kl(n), k2(n), k3(n), k4(n), f(n), dydx(n) ; 

// Zeroth intermediate step 
(*rhs_eval) (x, y, dydx) ; 
for (int j = 0; j < n; 
{ 

kl(j) = h * dydx(j); 

f (j) = y(j) + kl(j) / 2. ; 

} 

// First intermediate step 
(*rhs_eval) (x + h / 2., f, dydx); 
for (int j = 0; j < n; 
{ 

k2(j) = h * dydx(j); 

f (j) = y(j) + k2(j) / 2. ; 

} 

// Second intermediate step 
(*rhs_eval) (x + h / 2., f, dydx); 
for (int j = 0; j < n; j++) 
{ 

k3(j) = h * dydx(j); 
f(j) = y(j) + k3(j); 

} 

// Third intermediate step 
(*rhs_eval) (x + h, f, dydx); 
for (int j = 0; j < n; j++) 
{ 

k4(j) = h * dydx(j); 

} 

// Actual step 
for (int j = 0; j < n; j++) 
{ 

y(j) += kl(j) / 6. + k2(j) / 3. + k3(j) / 3. + k4(j) / 6.; 

} 

x += h; 
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return; 



} 



3.7 An Example Calculation 



Consider the following system of o.d.e.s: 



dx 
dt 
dv 
dt 



v 



— kx 



(3.29) 



(3.30) 



subject to the initial conditions x(0) = and v(0) = at t = 0. In fact, this 
system can be solved analytically to give 



Let us compare the above solution with that obtained numerically using either 
Euler's method or a fourth-order Runge-Kutta method. Figure [5] shows the inte- 
gration errors associated with these two methods (calculated by integrating the 
above system, with k = 1 , from t = to t = 1 0, and then taking the difference 
between the numerical and analytic solutions) plotted against the step-length, 
h, in a log-log graph. All calculations are performed to single precision: i.e., by 
using float, rather than double, variables. It can be seen that at large values 
of h, the error associated with Euler's method becomes much greater than unity 
(i.e., the magnitude of the numerical solution greatly exceeds that of the analytic 
solution), indicating the presence of a numerical instability. There are no simi- 
lar signs of instability associated with the Runge-Kutta method. At intermediate 
h, the error associated with Euler's method decreases smoothly like h 1 : in this 
regime, the dominant error is truncation error, which is expected to scale like 
h 1 for a first-order method. The error associated with the Runge-Kutta method 
similarly scales like H 4 — as expected for a fourth-order scheme — in the trunca- 
tion error dominated regime. Note that, as h is decreased, the error associated 
with both methods eventually starts to rise in a jagged curve that scales roughly 
like h 1 . This is a manifestation of round-off error. The minimum error associated 



x = sin(v / kt). 



(3.31) 
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log 10 (l/h) 

Figure 5: Global integration errors associated with Euler's method (solid curve) and a fourth-order 
Runge-Kutta method (dotted curve) plotted against the step-length h. Single precision calculation. 

with both methods corresponds to the boundary between the truncation error 
and round-off error dominated regimes. Thus, for Euler's method the minimum 
error is about 1 3 at h ~ 1 3 , whereas for the Runge-Kutta method the mini- 
mum error is about 1 5 at h ~ 1 1 . Clearly, the performance of the Runge-Kutta 
method is vastly superior to that of Euler's method, since the former method is 
capable of attaining much greater accuracy than the latter using a far smaller 
number of steps (i.e., a far larger H). 

Figure [6] displays similar data to that shown in Fig. [51 except that now all 
of the calculations are performed to double precision. The figure exhibits the 
same broad features as those apparent in Fig. O The major difference is that the 
round-off error has been reduced by about nine orders of magnitude, allowing 
the Runge-Kutta method to attain a minimum error of about 10 12 (see Tab. [ED — 
a remarkably performance! 

Figures [5] and [6] illustrate why scientists rarely use Euler's method, or single 
precision numerics, to integrate systems of o.d.e.s. 
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Figure 6: Global integration errors associated with Euler's method (solid curve) and a fourth-order 
Runge-Kutta method (dotted curve) plotted against the step-length h. Double precision calculation. 



3.8 Adaptive Integration Methods 



Consider the following system of o.d.e.s: 

(3.32) 

4kt 2 x, (3.33) 

subject to the boundary conditions x = Vk-y 2 and v = 2 Vk-y at x = y, where 
< -v <C 1 . This system can be solved analytically to give 

x = sin(v / kt 2 ). (3.34) 

One peculiarity of the above solution is that its variation scale-length decreases 
rapidly as t increases. 

Let us compare the above solution with that obtained numerically using a 
fourth-order Runge-Kutta method. Figure [7] shows the integration error asso- 
ciated with such a method (calculated by integrating the above system, with 
k = 10, from t = 10 3 to t = t, and then taking the difference between the 



dx 

dt = v ' 

dv v 

dt ~ t 
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Figure 7: Global integration error associated with a fixed step-length (h = 0.01 ), fourth-order 
Runge-Kutta method, plotted against the independent variable, t, for a system of o.d.e.s in which the 
variation scale-length decreases rapidly with increasing t. Double precision calculation. 



numerical and analytic solutions) with a fixed step-length of h = 0.01, plotted 
against the independent variable, t. It can be seen that, although the error starts 
off small, it rises rapidly as the variation scale-length of the solution decreases 
(i.e., as t increases), and quickly becomes unacceptably large. Of course, we 
could reduce the error by simply reducing the step-length, h. However, this is 
a very inefficient solution. The step-length only needs to be reduced at large t. 
There is no need to reduce it, at all, at small t. Clearly, the ideal solution to this 
problem would be an integration method in which the step-length is varied so as 
to maintain a relatively constant truncation error per step. Such an adaptive inte- 
gration method would take large steps when variation scale-length of the solution 
was large, and vice versa. 

Let us investigate how we could convert our fixed step-length, fourth-order 
Runge-Kutta method into a corresponding adaptive method. First of all, we need 
an estimate of the truncation error at each step. Suppose that the current step- 
length is H. We can estimate the truncation error, e, associated with the current 
step by taking the difference between the solutions obtained by stepping by h/2 
twice and by H once (starting from the same point, in both cases). Let eo be 
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the desired truncation error per step. How do we adjust h so as to ensure that 
the truncation error associated with the next step is closer to this value? Observe, 
from Eq. (13.281) . that the truncation error per step in a fourth-order scheme scales 
like h 5 . It follows, therefore, that our step-length adjustment formula should take 
the form@ 

'eo 



h-new — h-oid 



1/5 

(3.35) 



According to this formula, the step-length should be increased if the truncation 
error per step is too small, and vice versa, in such a manner that the error per step 
remains relatively constant at eo. 

There are a number of caveats to the above discussion. In a system of n 
coupled o.d.e.s, the overall truncation error per step, e, should, of course, be 
some appropriately weighted average of the errors associated with each equa- 
tion. There is also a question of whether e should be an absolute error or a 
relative error. The relative error associated with the ith equation is simply the 
absolute error divided by \yi\, where y\ is the current value of the ith dependent 
variable. An absolute error estimate is appropriate to a system of equations in 
which the amplitudes of the various dependent variables remain bounded. A rel- 
ative error estimate is appropriate to a system in which the amplitudes of the 
dependent variables blow-up at some point, but the variables always remain the 
same sign. Finally, a mixed error estimate — usually the minimum of the absolute 
and relative errors — is appropriate to a system in which the amplitudes of the 
dependent variables blow-up at some point, but the signs of the variables oscil- 
late. It is usually a good idea to place some limits on the allowed variation of 
the step-length from step to step: e.g., by preventing the step-length from in- 
creasing or decreasing by more than some factor S > 1 per step. This prevents 
h from oscillating unduly about its optimum value. Obviously, if h becomes ab- 
surdly small then the integration method has failed, and should abort with an 
appropriate error message. Finally, a limit should be placed on how large H can 
become — unfortunately, adaptive methods have a tendency to become a little 
over optimistic when integration is easy. 

^Numerical recipes in C: the art of scientific computing, W.H. Press, S.A. Teukolsky, W.T. Vettering, and B.R. Flannery 
(Cambridge University Press, Cambridge, England, 1992), p. 714. 
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2 4 6 8 10 
t 



Figure 8: Global integration errors associated with a fixed step-length (h = 0.01J, fourth-order 
Runge-Kutta method (solid curve) and a corresponding adaptive method (e Q = 1 8 J) (dotted curve), 
plotted against the independent variable, t, for a system of o.d.e.s in which the variation scale-length 
decreases rapidly with increasing t. Double precision calculation. 

Figure [8] shows the integration errors associated with a fixed step-length, fourth- 
order Runge-Kutta method and a corresponding adaptive method — constructed 
along the lines discussed above — as functions of the independent variable, t. 
The errors are calculated by integrating the current system, with k = 1 0, from 
t = 10 3 to t = t, and then taking the difference between the numerical and 
analytic solutions. The fixed step-length associated with the former method is 
h = 0.01. The desired truncation error per step associated with the latter is 
eo = 10 8 . It can be seen that the performance of the adaptive method is far su- 
perior to that of the fixed step-length method, since the former method maintains 
a relatively constant integration error as the variation scale-length of the solution 
decreases (i.e., as t increases). Figure [9] illustrates how this is achieved. This 
figure shows the step-length, h, associated with the adaptive method as a func- 
tion of t. It can be seen that the adaptive method maintains a relatively constant 
truncation error per step by decreasing h as t increases. 
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Figure 9: The step-length, K, associated with the adaptive integration method shown in the previous 
figure, plotted as a function of the independent variable, t. Double precision calculation. 

3.9 An Example Adaptive-Step RK4 Routine 

Listed below is an example adaptive-step RK4 routine which makes use of the 
previously listed fixed-step routine. Note that the routine recalculates all steps 
whose truncation error exceeds the desired value acc. 

/ / rk4_adaptive . cpp 
/* 

Function to advance set of coupled first-order o.d.e.s by single step 
using adaptive fourth-order Runge-Kutta scheme 



X 


. . . independent variable 


y 


. . . array of dependent variables 


h 


. . . step-length 


t_err 


. . actual truncation error per step 


acc 


. . desired truncation error per step 


S 


. . . step-length cannot change by more than this factor from 



rept 
maxrept 



step to step 
number of step recalculations 

maximum allowable number of step recalculations 



h_min 
h_max 



minimum allowable step-length 
maximum allowable step-length 
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flag . . . controls manner in which truncation error is calculated 

Requires right-hand side routine 

void rhs_eval (double x, Array<double , 1> y, Array<double , 1>& dydx) 

which evaluates derivatives of y (w.r.t. x) in array dydx. 

Function advances equations by single step whilst attempting to maintain 
constant truncation error per step of acc: 

flag = . . . error is absolute 
flag =1 ... error is relative 
flag =2 ... error is mixed 

If step-length falls below h_min then routine aborts 

*/ 

#include <stdio.h> 
#include <stdlib.h> 
#include <math.h> 
#include <blitz/array .h> 

using namespace blitz; 

void rk4_fixed (doublefe, Array<double, 1>&, 

void (*) (double, Array<double , 1> , Array<double , 1>&) , 
double) ; 

void rk4_adaptive (doublefe x, Array<double , 1>& y, 

void (*rhs_eval) (double , Array<double , 1> , Array<double , 1>&) , 
doublefe h, doublefe t_err, double acc, 
double S , int& rept , int maxrept , 
double h_min, double h_max, int flag) 

{ 

// Array y assumed to be of extent n, where n is no. of coupled 

// equations 

int n = y. extent (0); 

// Declare local arrays 
Array<double, 1> y0(n) , yl(n); 

// Declare repetition counter 
static int count = 0; 



118 



3.9 An Example Adaptive-Step RK4 Routine 



3 INTEGRATION OF ODES 



II Save initial data 
double xO = x; 

yo = y; 

// Take full step 

rk4_fixed (x, y, rhs_eval, h) ; 

// Save data 

yi = y; 

// Restore initial data 
x = xO; 

y = yo; 

// Take two half-steps 

rk4_fixed (x, y, rhs_eval, h/2.); 

rk4_fixed (x, y, rhs_eval, h/2.); 

// Calculate truncation error 
t_err = . ; 

double err, errl, err2; 
if (flag == 0) 
{ 

// Use absolute truncation error 
for (int i = 0; i < n; i++) 
{ 

err = fabs (y(i) - yl(i)); 

t_err = (err > t_err) ? err : t_err; 

> 

} 

else if (flag == 1) 
{ 

// Use relative truncation error 
for (int i = 0; i < n; i++) 
{ 

err = fabs ((y(i) - yl(i)) / y(i)); 
t_err = (err > t_err) ? err : t_err; 

> 

} 

else 
{ 

// Use mixed truncation error 
for (int i = 0; i < n; i++) 
{ 

errl = fabs ((y(i) - yl(i)) / y(i)); 
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err2 = fabs (y(i) - yl(i)); 

err = (errl < err2) ? errl : err2; 

t_err = (err > t_err) ? err : t_err; 

> 

} 

// Prevent small truncation error from rounding to zero 
if (t_err ==0.) t_err = l.e-15; 

// Calculate new step-length 

double h_est = h * pow (fabs (acc / t_err) , 0.2); 

// Prevent step-length from changing by more than factor S 
if (h_est / h > S) 
h *= S; 

else if (h_est / h < 1. / S) 

h /= S; 
else 

h = h_est; 

// Prevent step-length from exceeding h_max 

h = (fabs(h) > h_max) ? h_max * h / fabs(h) : h; 

// Abort if step-length falls below h_min 
if (fabs(h) < h_min) 
{ 

printf ("Error - |h| < hmin\n") ; 
exit (1); 

} 

// If truncation error acceptable take step 
if ((t_err <= acc) | | (rept >= maxrept)) 
{ 

rept = count; 
count = 0; 

} 

// If truncation error unacceptable repeat step 
else 
{ 

count++ ; 

x = xO; 

y = yO; 

rk4_adaptive (x, y, rhs_eval, h, t_err, acc, 

S, rept, maxrept, h_min, h_max, flag); 

} 
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return; 

> 



3.10 Advanced Integration Methods 

Of course, Runge-Kutta methods are not the last word in integrating o.d.e.s. Far 
from it! Runge-Kutta methods are sometimes referred to as single-step methods, 
since they evolve the solution from x n to x n+ i without needing to know the so- 
lutions at x n i, x n -z, etc. There is a broad class of more sophisticated integration 
methods, known as multi-step methods, which utilize the previously calculated 
solutions at x n i, x n _2, etc. in order to evolve the solution from x n to x n+ i. Exam- 
ples of these methods are the various Adams methods^ and the various Predictor- 
Corrector methods]^ The main advantages of Runge-Kutta methods are that they 
are easy to implement, they are very stable, and they are "self-starting" (i.e., un- 
like muti-step methods, we do not have to treat the first few steps taken by a 
single-step integration method as special cases). The primary disadvantages of 
Runge-Kutta methods are that they require significantly more computer time than 
multi-step methods of comparable accuracy, and they do not easily yield good 
global estimates of the truncation error. However, for the straightforward dy- 
namical systems under investigation in this course, the advantage of the relative 
simplicity and ease of use of Runge-Kutta methods far outweighs the disadvan- 
tage of their relatively high computational cost. 



3.11 The Physics of Baseball Pitching 

Baseball is the oldest professional sport in the U.S. It is a game of great subtlety 
(like cricket!) which has fascinated fans for over a hundred years. It has also 
fascinated physicists — partly, because many physicists are avid baseball fans, but, 
partly, also, because there are clearly delineated physics principles at work in 

17 Numerical methods, R.W. Hornbeck (Prentice-Hall, Englewood Cliffs NJ, 1975), p. 196. 
18 ibid, p. 199. 
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this game. Indeed, more books and papers have been written on the physics of 
baseball than on any other sport. 

A baseball is formed by winding yarn around a small sphere of cork. The ball 
is then covered with two interlocking pieces of white cowhide, which are tightly 
stitched together. The mass and circumference of a regulation baseball are 5 oz 
and 9 in (i.e., about 150 g and 23 cm), respectively. In the major leagues, the ball is 
pitched a distance of 60 feet 6 inches (i.e., 18.44 m), towards the hitter, at speeds 
which typically lie in the range 60 to 100 mph (i.e., about 30 to 45 m/s). As is 
well-known to baseball fans, there are a surprising variety of different pitches. 
"Sliders" deviate sideways through the air. "Curveballs" deviate sideways, but 
also dip unusually rapidly. Conversely, "fastballs" dip unusually slowly. Finally, 
the mysterious "knuckleball" can weave from side to side as it moves towards the 
hitter. How is all this bizarre behaviour possible? Let us investigate. 



3.12 Air Drag 

A baseball in flight is subject to three distinct forces. The first is gravity, which 
causes the ball to accelerate vertically downwards at g = 9.8 m/s . The second 
is air drag, which impedes the ball's motion through the air. The third is the 
Magnus force, which permits the ball to curve laterally. Let us discuss the latter 
two forces in more detail. 

As is well-known, the drag force acting on an object which moves very slowly 
through a viscous fluid is directly proportional to the speed of that object with 
respect to the fluid. For example, a sphere of radius r, moving with speed v 
through a fluid whose coefficient of viscosity is r\, experiences a drag force given 
by Stokes' lawj^ 

fo = 6nr\ rv. (3.36) 

As students who have attempted to reproduce Millikan's oil drop experiment will 
recall, this force is the dominant drag force acting on a microscopic oil drop 
falling through air. However, for most macroscopic projectiles moving through 



19 Methods of theoretical physics, Vol. II, P.M. Morse, and H. Feshbach, (McGraw-Hill, New York NY, 1953). 
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air, the above force is dwarfed by a second drag force which is proportional to v 2 . 

The origin of this second force is fairly easy to understand. At velocities suf- 
ficiently low for Stokes' law to be valid, air is able to flow smoothly around a 
passing projectile. However, at higher velocities, the projectile's motion is too 
rapid for this to occur. Instead, the projectile effectively knocks the air out of its 
way. The total mass of air which the projectile comes into contact with per second 
is p v A, where p is the air density, v the projectile speed, and A the projectile's 
cross-sectional area. Suppose, as seems reasonable, that the projectile imparts to 
this air mass a speed v' which is directly proportional to v. The rate of momen- 
tum gain of the air, which is equal to the drag force acting on the projectile, is 
approximately 

f D = ^C D (v)pAv 2 , (3.37) 
where the drag coefficient, Cd(v), is a dimensionless quantity. 

The drag force acting on a projectile, passing through air, always points in the 
opposite direction to the projectile's instantaneous direction of motion. Thus, the 
vector drag force takes the form 

f D = -^C D (v)pAvv. (3.38) 



When a projectile moves through air it leaves a turbulent wake. The value of 
the drag coefficient, Co, is closely related to the properties of this wake. Turbu- 
lence in fluids is conventionally characterized in terms of a dimensionless quan- 
tity known as a Reynolds number^ 

K = (3.39) 

Tl 

Here, d is the typical length-scale (e.g., the diameter) of the projectile. For suffi- 
ciently small Reynolds numbers, the air flow immediately above the surface of the 
projectile remains smooth, despite the presence of the turbulent wake, and Co 
takes an approximately constant value which depends on the projectile's shape. 
However, above a certain critical value of R e — which corresponds to 2 x 1 5 for 

20 R.E. Reynolds, Phil. Trans. Roy. Soc. 174, 935 (1883). 
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Figure 10: Typical dependence of the drag coefficient, C Dj , on the Reynolds number, R e . 

a smooth projectile — the air flow immediately above the surface of the projectile 
becomes turbulent, and the drag coefficient drops: i.e., a projectile slips more 
easily through the air when the surrounding flow is completely turbulent. In this 
high Reynolds number regime, the drag coefficient generally falls rapidly by a 
factor of between 3 and 10, as R e is increased, and then settles down to a new, 
roughly constant value. Note that the critical Reynolds number is significantly 
less than 2 x 10 5 for projectiles with rough surfaces. Paradoxically, a rough pro- 
jectile generally experiences less air drag, at high velocities, than a smooth one. 
The typical dependence of the drag coefficient on the Reynolds number is illus- 
trated schematically in Fig. ITOl 

Wind tunnel measurements reveal that the drag coefficient is a strong function 
of speed for baseballs, as illustrated in Fig. [Til At low speeds, the drag coefficient 
is approximately constant. However, as the speed increases, Cq drops substan- 
tially, and is more than a factor of 2 smaller at high speeds. This behaviour is 
similar to that described above. The sudden drop in the drag coefficient is trig- 
gered by a transition from laminar to turbulent flow in the air layer immediately 
above the ball's surface. The critical speed (to be more exact, the critical Reynolds 
number) at which this transition occurs depends on the properties of the surface. 
Thus, for a completely smooth baseball the transition occurs at speeds well be- 
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Figure 11: Variation of the drag coefficient, C Dj , with speed, v, for normal, rough, and smooth 
baseballs. From The physics of baseball, R.K. Adair (Harper & Row, New York NY, 1990). 

yond the capabilities of the fastest pitchers. Conversely, the transition takes place 
at comparatively low speeds if the surface of the ball is rough. In fact, the raised 
stitches on the otherwise smooth regulation baseball cause the transition to occur 
at an intermediate speed which is easily within the range of major league pitch- 
ers. Note that the magnitude of the drag force is substantial — it actually exceeds 
the force due to gravity for ball speeds above about 95 mph. 

The above discussion leads to a number of interesting observations. First, 
the raised stitches on a baseball have an important influence on its aerodynamic 
properties. Without them, the air drag acting on the ball at high speeds would 
increase substantially. Indeed, it seems unlikely that major league pitchers could 
throw 95 mph fastballs if baseballs were completely smooth. On the other hand, a 
scuffed-up baseball experiences even less air drag than a regulation ball. Presum- 
ably, such a ball can be thrown faster — which explains why balls are so regularly 
renewed in major league games. 

Giordano^ has developed the following useful formula which quantifies the 

21 Computational physics, N.J. Giordano, (Prentice-Hall, Upper Saddle River NJ, 1997). 
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drag force acting on a baseball: 

- = -F(v)vv, (3.40) 
m 

where 

t-, a /\ A/x ™ 0.0058 

F v = 0.0039 + — — - . (3.41) 

1 +exp[(v-v d )/A] 

Here, v d = 35 m/s and A = 5 m/s. 



3.13 The Magnus Force 

We have not yet explained how a baseball is able to curve through the air. Physi- 
cists in the last century could not account for this effect, and actually tried to 
dismiss it as an "optical illusion." It turns out that this strange phenomenon is 
associated with the fact that the balls thrown in major league games tend to spin 
fairly rapidly — typically, at 1500 rpm. 

The origin of the force which makes a spinning baseball curve can readily be 
appreciated once we recall that the drag force acting on a baseball increases with 
increasing speed@ For a ball spinning about an axis perpendicular to its direction 
of travel, the speed of the ball, relative to the air, is different on opposite sides of 
the ball, as illustrated in Fig. [121 It can be seen, from the diagram, that the lower 
side of the ball has a larger speed relative to the air than the upper side. This 
results in a larger drag force acting on the lower surface of the ball than on the 
upper surface. If we think of drag forces as exerting a sort of pressure on the ball 
then we can readily appreciate that when the unequal drag forces acting on the 
ball's upper and lower surfaces are added together there is a component of the 
resultant force acting upwards. This force — which is known as the Magnus force, 
after the German physicist Heinrich Magnus, who first described it in 1853 — is 
the dominant spin-dependent force acting on baseballs. The Magnus force can be 
written 

f M = S(v)a)xv, (3.42) 

22 Although the drag coefficient, Cd, decreases with increasing speed, the drag force, which is proportional to 
Cq v 2 , always increases. 
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Figure 12: Origin of the Magnus force for a ball of radius r, moving with speed v, and spinning with 
angular velocity cu about an axis perpendicular to its direction of motion. 

where cu is the angular velocity vector of the ball. According to Adair and Gior- 
dano, it is a fairly good approximation to take 

B = — = 4.1 x 10 4 (3.43) 
m 

for baseballs. Note that B is a dimensionless quantity. The magnitude of the 
Magnus force is about one third of the force due to gravity for typical curveballs. 



3.14 Simulations of Baseball Pitches 



Let us adopt a set of coordinates such that x measures displacement from the 
pitcher to the hitter, y measures horizontal displacement (a displacement in the 
+y direction corresponds to a displacement to the hitter's right-hand side), and z 
measures vertical displacement (a displacement in the +z direction corresponds 
to an upward displacement) . Using these coordinates, the equations of motion of 
a baseball can be written as the following set of coupled first-order o.d.e.s: 

dx 

(3.44) 



dt 
dt 



= v 



X) 



= V 



y > 



(3.45) 
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dz ,„ 

-77 = v z> (3.46) 
dt 

C ^ = — F(v) vv x + B cu (v z sincj) — v y cosc|)), (3.47) 

^ = -F(v) vv, + Bcuv x cos 4), (3.48) 
dt 

C ^ z = — g — F(v) vv z — B uov x sincj). (3.49) 
dt 

Here, the ball's angular velocity vector has been written cu = cu (0, sin c(), cos (J)). 
Appropriate boundary conditions at t = are: 

= 0, (3.50) 

= 0, (3.51) 

= 0, (3.52) 

= v cos (3.53) 

= 0, (3.54) 

= v sin0, (3.55) 

where vo is the initial speed of the pitch, and is its initial angle of elevation. 
Note that, in writing the above equations, we are neglecting any decrease in the 
ball's rate of spin as it moves towards the hitter. 

The above set of equations have been solved numerically using a fixed step- 
length, fourth-order Runge-Kutta method. The step-length, h, is conveniently 
expressed as a fraction of the pitch's estimated time-of- flight, T = 1/vq, where 
I = 18.44 m is the horizontal distance between the pitcher and the hitter. Thus, 
1/h is approximately the number of steps used to integrate the trajectory. 

It is helpful, at this stage, to describe the conventional classification of baseball 
pitches in terms of the direction of the ball's axis of spin. Figure [13] shows the 
direction of rotation of various different pitches thrown by a right-handed pitcher, 
as seen by the hitter. Obviously, the directions are reversed for the corresponding 
pitches thrown by a left-handed pitcher. Note, from Eq. (13.42D , that the arrows in 
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Fig. [13] show both the direction of the ball's spin and the direction of the Magnus 
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fastball curveball slider screwball 



Figure 13: Rotation direction, as seen by hitter, for various pitches thrown by a right-handed pitcher. 
The arrow shows the direction of rotation, which is also the direction of the Magnus force. From The 
physics of baseball, R.K. Adair (Harper & Row, New York NY, 1990). 

force. 

Figure [14] shows the numerical trajectory of a "slider" delivered by a right- 
handed pitcher. The ball is thrown such that its axis of rotation points vertically 
upwards, which is the direction of spin generated by the natural clockwise (seen 
from below) rotation of a right-handed pitcher's wrist. The associated Magnus 
force causes the ball to curve sideways, away from a right-handed hitter (i.e., in 
the +y -direction). As can be seen, from the figure, the sideways displacement 
for an 85mph pitch spinning at 1800rpm is over 1 foot. Of course, a slider 
delivered by a left-handed pitcher would curve towards a right-handed hitter. 
It turns out that pitches which curve towards a hitter are far harder to hit than 
pitches which curve away. Thus, a right-handed hitter is at a distinct disadvantage 
when facing a left-handed pitcher, and vice versa. A lot of strategy in baseball 
games is associated with teams trying to match the handedness of hitters and 
pitchers so as to gain an advantage over their opponents. 

Figure [14] shows the numerical trajectory of a "curveball" delivered by a right- 
handed pitcher. The ball is thrown such that its axis of rotation, as seen by the 
hitter, points upwards, but also tilts to the right. The hand action associated with 
throwing a curveball is actually somewhat more natural than that associated with 
a slider. The Magnus force acting on a curveball causes the ball to deviate both 
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Figure 14: Numerical trajectory of a slider delivered by a right-handed pitcher. The solid and dashed 
curves show the vertical and horizontal displacements of the ball, respectively. The parameters for 
this pitch are v = 85 mph, = 1 °, cu = 1 800 rpm, § = 0°, and h = 1 x 1 0~ 4 . The ball passes over 
the plate at 76 mph. about 0.52 seconds after it is released by the pitcher. 
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x (ft) 

Figure 15: Numerical trajectory of a curveball delivered by a right-handed pitcher. The solid and 
dashed curves show the vertical and horizontal displacements of the ball, respectively. The parameters 
for this pitch are v = 85 mph, 8 = 1 °, cu = 1 800 rpm, § = 45°, and h = 1 x 1 0~ 4 . The ball passes 
over the plate at 76 mph. about 0.52 seconds after it is released by the pitcher. 

sideways and downwards. Thus, although a curveball generally does not move 
laterally as far as a slider, it dips unusually rapidly — which makes it difficult to hit. 
The anomalously large dip of a typical curveball is apparent from a comparison 
of Figs. [H and US 

As we have already mentioned, a pitch which curves towards a hitter is harder 
to hit than one which curves away. It is actually possible for a right-handed 
pitcher to throw an inward curving pitch to a right-handed hitter. Such a pitch is 
known as a "screwball." Unfortunately, throwing screwballs involves a completely 
unnatural wrist rotation which is extremely difficult to master. Figure [[6] shows 
the numerical trajectory of a screwball delivered by a right-handed pitcher. The 
ball is thrown such that its axis of rotation, as seen by the hitter, points upwards 
and tilts to the left. Note that the pitch dips rapidly — like a curveball — but has a 
sideways displacement in the opposite direction to normal. 

Probably the most effective pitch in baseball is the so-called "fastball." As the 
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Figure 16: Numerical trajectory of a screwball delivered by a right-handed pitcher. The solid and 
dashed curves show the vertical and horizontal displacements of the ball, respectively. The parameters 
for this pitch are v = 85 mph, = 1 °, tu = 1 800 rpm, § = 1 35°, and h = 1 x 1 0~ 4 . The ball passes 
over the plate at 76 mph about 0.52 seconds after it is released by the pitcher. 
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Figure 17: Numerical trajectory of a fastball delivered by a right-handed pitcher. The solid and 
dashed curves show the vertical and horizontal displacements of the ball, respectively. The parameters 
for this pitch are v = 95 mph, 9 = 1 °, cu = 1 800 rpm, § = 225°, and h = 1 x 1 0~ 4 . The ball passes 
over the plate at 86 mph. about 0.46 seconds after it is released by the pitcher. 



name suggests, a fastball is thrown extremely rapidly — typically, at 95 mph. The 
natural hand action associated with throwing this type of pitch tends to impart 
a significant amount of backspin to the ball. Thus, the Magnus force acting on a 
fastball has a large upwards component, slowing the ball's rate of fall. Figure [T71 
shows the numerical trajectory of a fastball delivered by a right-handed pitcher. 
The ball is thrown such that its axis of rotation, as seen by the hitter, points down- 
wards and tilts to the left. Note that the pitch falls an unusually small distance. 
This is partly because the ball takes less time than normal to reach the hitter, but 
partly also because the Magnus force has a substantial upward component. This 
type of pitch is often called a "rising fastball," because hitters often claim that the 
ball rises as it moves towards them. Actually, this is an optical illusion created by 
the ball's smaller than expected rate of fall. 
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3.15 The Knuckleball 



Probably the most entertaining pitch in baseball is the so-called "knuckleball." 
Unlike the other types of pitch we have encountered, knuckleballs are low speed 
pitches (typically 65 mph) in which the ball is purposely thrown with as little spin 
as possible. It turns out that knuckleballs are hard to hit because they have un- 
stable trajectories which shift from side to side in a highly unpredictable manner. 
How is this possible? 

Suppose that a moving ball is not spinning at all, and is orientated such that 
one of its stitches is exposed on one side, whereas the other side is smooth. It 
follows, from Fig. [EH that the drag force on the smooth side of the ball is greater 
than that on the stitch side. Hence, the ball experiences a lateral force in the 
direction of the exposed stitch. Suppose, now, that the ball is rotating slowly as it 
moves towards the hitter. As the ball moves forward its orientation changes, and 
the exposed stitch shifts from side to side, giving rise to a lateral force which also 
shifts from side to side. Of course, if the ball is rotating sufficiently rapidly then 
the oscillations in the lateral force average out. However, for a slowly rotating 
ball these oscillations can profoundly affect the ball's trajectory. 



Watts and Sawyer^ have performed wind tunnel measurements of the lateral 
force acting on a baseball as a function of its angular orientation. Note that the 
stitches on a baseball pass any given point four times for each complete revolution 
of the ball about an axis passing through its centre. Hence, we expect the lateral 
force to exhibit four maxima and four minima as the ball is rotated once. This is 
exactly what Watts and Sawyer observed. For the case of a 65 mph knuckleball, 
Giordano has extracted the following useful expression for the lateral force, as a 
function of angular orientation, (p, from Watts and Sawyer's data: 



: G((p) = 0.5 [sin(4(p) - 0.25 sin(8cp) + 0.08 sin(12<p) - 0.025 sin(16cp)] . 
m g 

(3.56) 



The function G(cp) is plotted in Fig. [18]. Note the very rapid changes in this 
function in certain narrow ranges of (p. 



23 



Aerodynamics of a knuckleball, R.G. Watts, and E. Sawyer, Am. J. of Phys. 43, 960 (1975). 
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Figure 18: The lateral force function, G{<p), for a 65mph knuckleball. 

Using the above expression, the equations of motion of a knuckleball can be 
written as the following set of coupled first-order o.d.e.s: 

dx 



dt 
dy 
dt 
dz 
dt 
dv x 
dt 

dVy 

dt 

dv z 

~dt 
d(p 
dt 



= v 



X) 



= V 



-F(v)vv x , 

-F(v) vv y + g G(cp), 

-g-F(v)vv z . 



= cu. 



(3.57) 
(3.58) 
(3.59) 
(3.60) 
(3.61) 
(3.62) 
(3.63) 



Note that we have added an equation of motion for the angular orientation, <p, 
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of the ball, which is assumed to rotate at a fixed angular velocity, cu, about a 
vertical axis. Here, we are neglecting the Magnus force, which is expected to be 
negligibly small for a slowly spinning pitch. Appropriate boundary conditions at 
t = are: 



x(t = 0) 


= o, 


(3.64) 


v(t = o) 


= o, 


(3.65) 


z(t = 0) 


= o, 


(3.66) 


v x (t = 0) 


= v cos 


(3.67) 


v y (t = 0) 


= o, 


(3.68) 


v z (t = 0) 


= v sin0, 


(3.69) 


<p(t = 0) 


= <Po, 


(3.70) 



where vo is the initial speed of the pitch, is the pitch's initial angle of elevation, 
and (po is the ball's initial angular orientation. 

The above set of equations have been solved numerically using a fixed step- 
length, fourth-order Runge-Kutta method. The step-length, h, is conveniently 
expressed as a fraction of the pitch's estimated time-of- flight, T = X/vq, where 
I = 18.44 m is the horizontal distance between the pitcher and the hitter. Thus, 
1/h is approximately the number of steps used to integrate the trajectory. 

Figure [19] shows the lateral displacement of a set of four knuckleballs thrown 
with the same rate of spin, cu = 20 rpm, but starting with different angular ori- 
entations. Note the striking difference between the various trajectories shown in 
this figure. Clearly, even a fairly small change in the initial orientation of the ball 
translates into a large change in its subsequent trajectory through the air. For this 
reason, knuckleballs are extremely unpredictable — neither the pitcher, the hitter, 
nor the catcher can be really sure where one of these pitches is going to end up. 
Needless to say, baseball teams always put their best catcher behind the plate 
when a knuckleball pitcher is on the mound! 

Figure [20] shows the lateral displacement of a knuckleball thrown with a some- 
what higher rate of spin than those shown previously: i.e., cu =40 rpm. Note the 
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Figure 19: Numerical trajectories of knuckleballs with the same angular velocity but different initial 
orientations. The solid, dotted, long-dashed, and short-dashed curves show the horizontal displace- 
ment of the ball for cp = 0°, 22.5°, 45°, and 67.5°, respectively. The other parameters for this pitch 
are v = 65 mph, 9 = 4°, cu = 20 rpm, and h = 1 x 1 0~ 4 . The ball passes over the plate at 56 mph. 
about 0.69 seconds after it is released by the pitcher. The ball rotates about 83° whilst it is in the air. 
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Figure 20: Numerical trajectory of a knuckleball The parameters for this pitch are v = 65mph, 
9 = 4°, (p = 0°, cu = 40 rpm, and h = 1 x 1 0~ 4 . The ball passes over the plate at 56 rap h, about 0.69 
seconds after it is released by the pitcher. The ball rotates about 1 66° whilst it is in the air. 
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curious way in which the ball "dances" through the air. Given that the difference 
between a good hit and a bad hit can correspond to a shift in the strike point on 
the bat by as little as 1 /4 of an inch, it must be quite a challenge to hit such a 
pitch well! 
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4 The Chaotic Pendulum 



4. 1 Introduction 



Up to now, we have mostly dealt with problems which are capable of analytic 
solution (so that we can easily validate our numerical solutions). Let us now 
investigate a problem which is quite intractable analytically and in which mean- 
ingful progress can only be made via numerical means. 

Consider a simple pendulum consisting of a point mass m, at the end of a 
light rigid rod of length I, attached to a fixed frictionless pivot which allows the 
rod (and the mass) to move freely under gravity in the vertical plane. Such a 
pendulum is sketched in Fig. [211 Let us parameterize the instantaneous position 
of the pendulum via the angle the rod makes with the downward vertical. It is 
assumed that the pendulum is free to swing through 360 degrees. Hence, and 
+ 2n both correspond to the same pendulum position. 

The angular equation of motion of the pendulum is simply 

d 2 

+ mg sin0 = 0, (4.1) 

where g is the downward acceleration due to gravity. Suppose that the pendulum 
is embedded in a viscous medium (e.g., air). Let us assume that the viscous drag 
torque acting on the pendulum is governed by Stokes' law (see Sect. 13.121) and 
is, thus, directly proportional to the pendulum's instantaneous velocity. It follows 
that, in the presence of viscous drag, the above equation generalizes to 

d 2 d0 

ml— r +-v — + mg sin0 = 0, (4.2) 
dt z dt 

where y is a positive constant parameterizing the viscosity of the medium in 
question. Of course, viscous damping will eventually drain all energy from the 
pendulum, leaving it in a stationary state. In order to maintain the motion against 
viscosity, it is necessary to add some external driving. For the sake of simplicity, 
we choose a fixed amplitude, periodic drive (which could arise, for instance, via 
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Figure 21: A simple pendulum. 

periodic oscillations of the pendulum's pivot point). Thus, the final equation of 
motion of the pendulum is written 



d 2 e de 

m I — j + -v - — hmg sin0 = A cos cut, 
dt 2 dt 



(4.3) 



where A and eo are constants parameterizing the amplitude and angular fre- 
quency of the external driving torque, respectively. 



Let 



cu 



(4.4) 



Of course, we recognize cu as the natural (angular) frequency of small amplitude 
oscillations of the pendulum. We can conveniently normalize the pendulum's 
equation of motion by writing, 

(4.5) 
(4.6) 

(4.7) 
(4.8) 
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in which case Eq. (14.31) becomes 

d 2 9 1 d9 . n 

-Tvy + — + sin 9 = A cos d)t. (4.9) 
dt z Q dt 

From now on, the hats on normalized quantities will be omitted, for ease of no- 
tation. Note that, in normalized units, the natural frequency of small amplitude 
oscillations is unity. Moreover, Q is the familiar quality-factor — roughly, the num- 
ber of oscillations of the undriven system which must elapse before its energy is 
significantly reduced via the action of viscosity. The quantity A is the amplitude 
of the external torque measured in units of the maximum possible gravitational 
torque. Finally, cu is the angular frequency of the external torque measured in 
units of the pendulum's natural frequency. 

Equation (14. 9D is clearly a second-order o.d.e. It can, therefore, also be written 
as two coupled first-order o.d.e.s: 



d9 
dt 

dv v 
dt = ~Q 



v, (4.10) 
sin + A cos cut. (4.11) 



4.2 Analytic Solution 

Before attempting to solve the equations of motion of any dynamical system us- 
ing a computer, we should, first, investigate them as thoroughly as possible via 
standard analytic techniques. Unfortunately, Eqs. (14.101) and (14.111) constitute a 



non-linear dynamical system — because of the presence of the sin term on the 
right-hand side of Eq. (14.111) . This system, like most non- linear systems, does 
not possess a simple analytic solution. Fortunately, however, if we restrict our 
attention to small amplitude oscillations, such that the approximation 

sin0~0 (4.12) 

is valid, then the system becomes linear, and can easily be solved analytically. 
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The linearized equations of motion of the pendulum take the form: 

d9 

dt = V ' 
dv v 



dt 



Q 



— + A cos cut. 



(4.13) 
(4.14) 



Suppose that the pendulum's position, 0(0), and velocity, v(0), are specified at 
time t = 0. As is well-known, in this case, the above equations of motion can be 
solved analytically to give: 



e(t) = 



9(0)- 

i 



A(1 -cu 2 ) 



-t/2Q 



[(1- 

v(0) + 



cu 2 ) 2 + cu 2 /Q 2 ] J 

0(0) _ A(1-3cu 2 )/2Q 
2Q~" [(1 - cu 2 ) 2 + cu 2 /Q 2 ] 



cos cu*t 



-t/2Q 



sin tu*t 



A [(1 — cu 2 ) cos cut + (cu/Q) sin cut 



v(t) 



v(0)- 
1 



[(1 - cu 2 ) 2 + cu 2 /Q 2 ] 
Acu 2 /Q 



(4.15) 



cu* 
cuA 



[(1- 
0(0) + 



| e t/2Q cos cu,t 



cu 2 ) 2 + cu 2 /Q 2 ] 
v(0) A[(l - cu 2 ) - cu 2 /2Q 2 ] 



-t/2Q 



2Q [(1 - cu 2 ) 2 + cu 2 /Q 2 ] 
(1 — cu 2 ) sin cut + (cu/Q) cos cut 



sin cu*t 



[(1 - cu 2 ) 2 + cu 2 /Q 2 ] 



Here, 



cu* = 



1 



V 4Q^ 



(4.16) 



(4.17) 



and it is assumed that Q > 1/2. It can be seen that the above expressions 
for and v both consist of three terms. The first two terms clearly represent 
transients — they depend on the initial conditions, and decay exponentially in 
time. In fact, the e-folding time for the decay of these terms is 2 Q (in nor- 
malized time units). The final term represents the time-asymptotic motion of the 
pendulum, and is manifestly independent of the initial conditions. 
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Figure 22: A phase-space plot of the periodic attractor for a linear, damped, periodically driven, 
pendulum. Data calculated analytically for Q = 4 and a> = 2. 

It is often convenient to visualize the motion of a dynamical system as an orbit, 
or trajectory in phase-space, which is defined as the space of all of the dynamical 
variables required to specify the instantaneous state of the system. For the case in 
hand, there are two dynamical variables, v and 0, and so phase-space corresponds 
to the 0-v plane. Note that each different point in this plane corresponds to a 
unique instantaneous state of the pendulum. [Strictly speaking, we should also 
consider t to be a dynamical variable, since it appears explicitly on the right-hand 
side of Eq. (143TD .1 



It is clear, from Eqs. (14.151) and (14.161) , that if we wait long enough for all of 



the transients to decay away then the motion of the pendulum settles down to 
the following simple orbit in phase-space: 

A [(1 — co 2 ) cos cut + (eu/Q) sincutl 
9 W = — 2UT 2/n2 i > W- 18 ) 

[(1 - CO ) + CU 2 /Q ] 

, , cuA [— (1 — co 2 ) sin cut + (eu/Q) cos cut 

V W = 1 rn 2Ux 2/n2i -• (4 ' 19 ) 

[(1 - co 1 ) 1 + cuVQ ] 
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This orbit traces out the closed curve 

G)"* (=*)'-■ 

in phase-space, where 

A = A (4.21) 

'(1 - CD 2 ) 2 + CD 2 /Q 2 



As illustrated in Fig. [221 this curve is an ellipse whose principal axes are aligned 
with the v and axes. Observe that the curve is closed, which suggests that the 
associated motion is periodic in time. In fact, the motion repeats itself exactly 
every 

x = 271 (4.22) 

CO 

normalized time units. The maximum angular displacement of the pendulum 
from its undriven rest position (0 = 0) is A. As illustrated in Fig. [23], the variation 



of A with driving frequency co [see Eq. (14. 211) 1 displays all of the features of a 
classic resonance curve. The maximum amplitude of the driven oscillation is 
proportional to the quality-factor, Q, and is achieved when the driving frequency 
matches the natural frequency of the pendulum (i.e., when |cu| = 1). Moreover, 
the width of the resonance in co -space is proportional to 1/Q. 

The phase-space curve shown in Fig. [22] is called a periodic attractor. It is 
termed an "attractor" because, irrespective of the initial conditions, the trajectory 
of the system in phase-space tends asymptotically to — in other words, is attracted 
to — this curve as t — > oo. This gravitation of phase-space trajectories towards the 
attractor is illustrated in Figs. [24] and [251 Of course, the attractor is termed 
"periodic" because it corresponds to motion which is periodic in time. 

Let us summarize our findings, so far. We have discovered that if a damped 
pendulum is subject to a low amplitude, periodic, drive then its time-asymptotic 
response (i.e., its response after any transients have died away) is periodic, with 
the same period as the driving torque. Moreover, the response exhibits resonant 
behaviour as the driving frequency approaches the natural frequency of oscilla- 
tion of the pendulum. The amplitude of the resonant response, as well as the 
width of the resonant window, is governed by the amount of damping in the sys- 
tem. After a little reflection, we can easily appreciate that all of these results are a 
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Figure 23: The maximum angular displacement of a linear, damped, periodically driven, pendulum 
as a function of driving frequency. The solid curve corresponds to Q = 1 . The short-dashed curve 
corresponds to Q = 5. The long-dashed curve corresponds to Q = 10. Analytic data. 




I A 



Figure 24: The phase-space trajectory of a linear, damped, periodically driven, pendulum. Data 
calculated analytically for Q = 1 and cu = 2. Here, v(0)/A = and 0(O)/A = 0. 
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Figure 25: The phase-space trajectory of a linear, damped, periodically driven, pendulum. Data 
calculated analytically for Q = 1 and cu = 2. Here, v(0)/A = 0.5 and 9(0)/A = 0.5. 

direct consequence of the linearity of the pendulum's equations of motion in the 
low amplitude limit. In fact, it is easily demonstrated that the time- asymptotic 
response of any intrinsically stable linear system (with a discrete spectrum of 
normal modes) to a periodic drive is periodic, with the same period as the drive. 
Moreover, if the driving frequency approaches one of the natural frequencies of 
oscillation of the system then the response exhibits resonant behaviour. But, is 
this the only allowable time-asymptotic response of a dynamical system to a pe- 
riodic drive? Most undergraduate students might be forgiven for answering this 
question in the affirmative. After all, the majority of undergraduate classical dy- 
namics courses focus almost exclusively on linear systems. The correct answer, 
as we shall see, is no. The response of a non-linear system to a periodic drive 
is generally far more rich and diverse than simple periodic motion. Since the 
majority of naturally occurring dynamical systems are non-linear, it is clearly im- 
portant that we gain a basic understanding of this phenomenon. Unfortunately, 
we cannot achieve this goal via a standard analytic approach — non-linear equa- 
tions of motion generally do not possess simple analytic solutions. Instead, we 
must use computers. As an example, let us investigate the dynamics of a damped 
pendulum, subject to a periodic drive, with no restrictions on the amplitude of the 
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pendulum's motion. 



4.3 Numerical Solution 



In the following, we present numerical solutions of Eqs. (14.101) and (14. IIP ob- 
tained using a fixed step-length, fourth-order, Runge-Kutta integration scheme. 
The step-length is conveniently parameterized by Nacc, which is defined as the 
number of time-steps taken by the integration scheme per period of the external 
drive. 



4.4 Validation of Numerical Solutions 

Before proceeding with our investigation, we must first convince ourselves that 
our numerical solutions are valid. Now, the usual method of validating a numer- 
ical solution is to look for some special limits of the input parameters for which 
analytic solutions are available, and then to test the numerical solution in one of 
these limits against the associated analytic solution. 



One special limit of Eqs. (I4.10D and (14. IIP occurs when there is no viscous 



damping (i.e., Q — > oo) and no external driving (i.e., A — > 0). In this case, we 
expect the normalized energy of the pendulum 

v 2 

8 = 1 + — - cos 9 (4.23) 

to be a constant of the motion. Note that 8 is defined such that the energy is 
zero when the pendulum is in its stable equilibrium state (i.e., at rest, pointing 
vertically downwards) . Figure [26] shows 8 versus time, calculated numerically 
for an undamped, undriven, pendulum. Curves are plotted for various values of 
the parameter Nacc, which, in this special case, measures the number of time- 
steps taken by the integrator per (low amplitude) natural period of oscillation of 
the pendulum. It can be seen that for Nacc = 12 there is a strong spurious loss 
of energy, due to truncation error in the numerical integration scheme, which 
eventually drains all energy from the pendulum after about 2000 oscillations. 
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Figure 26: The normalized energy £ of an undamped, undriven, pendulum versus time (measured 
in natural periods of oscillation r). Data calculated numerically for Q = 10 16 , A = 10~ 16 , cu = ], 
9(0) = 0, and v(0) = 1. The dotted curve shows data for Nacc = 12. The dashed curve shows data 
for Nacc = 24. The dot-dashed curve shows data for Nacc = 48. Finally, the solid curve shows data 
for Nacc = 96. 

For Nacc = 24, the spurious energy loss is less severe, but, nevertheless, still 
causes a more than 50% reduction in pendulum energy after 10,000 oscillations. 
For Nacc = 48, the reduction in energy after 10,000 oscillations is only about 
1%. Finally, for Nacc = 96, the reduction in energy after 10,000 oscillation is 
completely negligible. This test seems to indicate that when Nacc > 100 our 
numerical solution describes the pendulum's motion to a high degree of precision 
for at least 10,000 oscillations. 



Another special limit of Eqs. (14.101) and (14. IIP occurs when these equations 
are linearized to give Eqs. (14.131) and (I4.14D . In this case, we expect 



R = a/0 2 + (v/uo) 2 



(4.24) 



to be a constant of the motion, after all transients have died away (see Sect. 14. 2D . 
Figure [27] shows R versus time, calculated numerically, for a linearized, damped, 
periodically driven, pendulum. Curves are plotted for various values of the pa- 
rameter Nacc, which measures the number of time-steps taken by the integrator 
per period of oscillation of the external drive. As Nacc increases, it can be seen 



149 



4.4 Validation of Numerical Solutions 



4 THE CHAOTIC PENDULUM 



0.12287 



0.12286 



0.12285 H 



0.12284 



997 



998 



t / 



999 



1000 



Figure 27: The parameter R associated with a linearized, damped, periodically driven, pendulum 
versus time (measured in units of the period of oscillation T of the external drive). Data calculated 
numerically for Q = 2, A = \, cu = 3, 9(0) = 0, and v(0) = 0. The dotted curve shows data for 
Nacc = 12. The dashed curve shows data for Nacc = 24. The solid curve shows data for Nacc = 48. 



that the amplitude of the spurious oscillations in R, which are due to trunca- 
tion error in the numerical integration scheme, decreases rapidly. Indeed, for 
Nacc > 48 these oscillations become effectively undetectable. According to the 
analysis in Sect. 14.21 the parameter R should take the value 



R 



A 



(1 - CD 2 ) 2 + CD 2 /Q 2 



(4.25) 



Thus, for the case in hand (i.e., Q = 2, A = 1 , cu = 3), we expect R = 0.122859. 
It can be seen that this prediction is borne out very accurately in Fig. [27]. The 
above test essentially confirms our previous conclusion that when Nacc > 100 
our numerical solution matches pendulum's actual motion to a high degree of 
accuracy for many thousands of oscillation periods. 
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Figure 28: Equally spaced (in time) points on a time-asymptotic orbit in phase-space. Data calculated 
numerically for Q = 0.5, A = 1 .5, w = 2/3, 9(0) = 0, v(0) = 0, and Nacc = 100. 

4.5 The Poincare Section 



For the sake of definiteness, let us fix the normalized amplitude and frequency 
of the external drive to be A = 1.5 and cu = 2/3, respectivelyo Furthermore, 
let us investigate any changes which may develop in the nature of the pendu- 
lum's time-asymptotic motion as the quality-factor Q is varied. Of course, if Q 
is made sufficiently small (i.e., if the pendulum is embedded in a sufficiently vis- 
cous medium) then we expect the amplitude of the pendulum's time-asymptotic 
motion to become low enough that the linear analysis outlined in Sect. 14.21 re- 
mains valid. Indeed, we expect non-linear effects to manifest themselves as Q is 
gradually made larger, and the amplitude of the pendulum's motion consequently 
increases to such an extent that the small angle approximation breaks down. 



Figure [281 shows a time-asymptotic orbit in phase-space calculated numerically 
for a case where Q is sufficiently small (i.e., Q = 1/2) that the small angle ap- 
proximation holds reasonably well. Not surprisingly, the orbit is very similar to 
the analytic orbits described in Sect. 14.21 The fact that the orbit consists of a 
single loop, and forms a closed curve in phase-space, strongly suggests that the 



24 G.L. Baker, Control of the chaotic driven pendulum, Am. J. Phys. 63, 832 (1995). 
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corresponding motion is periodic with the same period as the external drive — we 
term this type of motion period- 1 motion. More generally, period-n motion con- 
sists of motion which repeats itself exactly every n periods of the external drive 
(and, obviously, does not repeat itself on any time-scale less than n periods) . Of 
course, period- 1 motion is the only allowed time- asymptotic motion in the small 
angle limit. 

It would certainly be helpful to possess a graphical test for period-n motion. 
In fact, such a test was developed more than a hundred years ago by the French 
mathematician Henry Poincare — nowadays, it is called a Poincare section in his 
honour. The idea of a Poincare section, as applied to a periodically driven pen- 
dulum, is very simple. As before, we calculate the time-asymptotic motion of the 
pendulum, and visualize it as a series of points in 0-v phase-space. However, we 
only plot one point per period of the external drive. To be more exact, we only 
plot a point when 

cut = ct) + k27t (4.26) 

where k is any integer, and c() is referred to as the Poincare phase. For period- 1 
motion, in which the motion repeats itself exactly every period of the external 
drive, we expect the Poincare section to consist of only one point in phase-space 
(i.e., we expect all of the points to plot on top of one another). Likewise, for 
period-2 motion, in which the motion repeats itself exactly every two periods 
of the external drive, we expect the Poincare section to consist of two points in 
phase-space (i.e., we expect alternating points to plot on top of one another). 
Finally, for period-n motion we expect the Poincare section to consist of n points 
in phase-space. 

Figure [29] displays the Poincare section of the orbit shown in Fig. 128]. The fact 
that the section consists of a single point confirms that the motion displayed in 
Fig. [28] is indeed period- 1 motion. 



4.6 Spatial Symmetry Breaking 



Suppose that we now gradually increase the quality- factor Q . What happens to 
the simple orbit shown in Fig. [28] ? It turns out that, at first, nothing particularly 
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Figure 29: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q = 0.5, 
A = 1 .5, cu = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, and $ = 0. 



2.4 - 



2.2 - 



> 2 - 



1.8 - 



1.6 - 




Figure 30: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1 .5, cv = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, 
and c() = 0. 
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exciting happens. The size of the orbit gradually increases, indicating a corre- 
sponding increase in the amplitude of the pendulum's motion, but the general 
nature of the motion remains unchanged. However, something interesting does 
occur when Q is increased beyond about 1 .2. Figure [30] shows the v-coordinate 
of the orbit's Poincare section plotted against Q in the range 1 .2 and 1 .3. Note the 
sharp downturn in the curve at Q ~ 1 .245. What does this signify? Well, Fig. I3T1 
shows the time- asymptotic phase-space orbit just before the downturn (i.e., at 
Q = 1.24), and Fig. [32] shows the orbit somewhat after the downturn (i.e., at 
Q = 1.30). It is clear that the downturn is associated with a sudden change 
in the nature of the pendulum's time-asymptotic phase-space orbit. Prior to the 
downturn, the orbit spends as much time in the region < as in the region 
> 0. However, after the downturn the orbit spends the majority of its time in 
the region < 0. In other words, after the downturn the pendulum bob favours 
the region to the left of the pendulum's vertical. This is somewhat surprising, 
since there is nothing in the pendulum's equations of motion which differentiates 
between the regions to the left and to the right of the vertical. We refer to a 
solution of this type — which fails to realize the full symmetry of the dynamical 
system in question — as a symmetry breaking solution. In this case, because the 
particular symmetry which is broken is a spatial symmetry, we refer to the pro- 
cess by which the symmetry breaking solution suddenly appears, as the control 
parameter Q is adjusted, as spatial symmetry breaking. Needless to say, spatial 
symmetry breaking is an intrinsically non-linear process — it cannot take place in 
dynamical systems possessing linear equations of motion. 

It stands to reason that since the pendulum's equations of motion favour nei- 
ther the left nor the right then the left- favouring orbit pictured in Fig. [32] must 
be accompanied by a mirror image right-favouring orbit. How do we obtain this 
mirror image orbit? It turns out that all we have to do is keep the physics param- 
eters Q, A, and co fixed, but change the initial conditions 0(0) and v(0). Figure [33] 
shows a time-asymptotic phase-space orbit calculated with the same physics pa- 
rameters used in Fig. [32], but with the initial conditions 0(0) = and v(0) = —3, 
instead of 0(0) =0 and v(0) =0. It can be seen that the orbit is indeed the mirror 
image of that pictured in Fig. [321 



Figure [341 shows the v-coordinate of the Poincare section of a time- asymptotic 
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Figure 31: Equally spaced (in time) points on a time-asymptotic orbit in phase-space. Data calculated 
numerically for Q = 1 .24, A = 1 .5, w = 2/3, 9(0) = 0, v(0) = 0, and Nacc = 100. 




Figure 32: Equally spaced (in time) points on a time-asymptotic orbit in phase-space. Data calculated 
numerically for Q = 1 .30, A = 1 .5, w = 2/3, 9(0) = 0, v(0) = 0, and Nacc = 100. 
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Figure 33: Equally spaced (in time) points on a time-asymptotic orbit in phase-space. Data calculated 
numerically for Q = 1 .30, A = 1 .5, w = 2/3, 9(0) = 0, v(0) = -3, and Nacc = 100. 



orbit, calculated with the same physics parameters used in Fig. [301 versus Q in the 
range 1 .2 and 1 .3. Data is shown for the two sets of initial conditions discussed 
above. The figure is interpreted as follows. When Q is less than a critical value, 
which is about 1 .245, then the two sets of initial conditions lead to motions which 
converge on the same, left-right symmetric, period- 1 attractor. However, once Q 
exceeds the critical value then the attractor bifurcates into two asymmetric, mirror 
image, period- 1 attractors. Obviously, the bifurcation is indicated by the forking 
of the curve shown in Fig. [34} The lower and upper branches correspond to the 
left- and right-favouring attractors, respectively. 

Spontaneous symmetry breaking, which is the fundamental non-linear pro- 
cess illustrated in the above discussion, plays an important role in many areas of 
physics. For instance, symmetry breaking gives mass to elementary particles in 
the unified theory of electromagnetic and weak interactions O Symmetry break- 
ing also plays a pivotal role in the so-called "inflation" theory of the expansion of 
the early universe 



26 



25 E.S. Albers and B.W. Lee, Phys. Rep. 9C, 1 (1973). 

26 P. Coles, and F. Lucchin, Cosmology: The origin and evolution of cosmic structure, (J. Wiley & Sons, Chichester 



UK, 1995). 
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Figure 34: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1 .5, cu = 2/3, and Nacc = 100. Data is shown 
for two sets of initial conditions: 9(0) =0 and v(0) = (lower branch); and 9(0) =0 and v(0) = —3 
(upper branch). 

4.7 Basins of Attraction 

We have seen that when Q = 1.3, A = 1.5, and cu = 2/3 there are two co- 
existing period- 1 attractors in 0-v phase-space. The time -asymptotic trajectory of 
the pendulum through phase-space converges on one or other of these attractors 
depending on the initial conditions: i.e., depending on the values of 0(0) and v(0) . 
Let us define the basin of attraction of a given attractor as the locus of all points 
in the 0(O)-v(O) plane which lead to motion which ultimately converges on that 
attractor. We have seen that in the low-amplitude (i.e., linear) limit (see Sect. 14.21) 
there is only a single period- 1 attractor in phase-space, and all possible initial 
conditions lead to motion which converges on this attractor. In other words, the 
basin of attraction for the low-amplitude attractor constitutes the entire 0(0)- 
v(0) plane. The present case, in which there are two co-existing attractors in 
phase-space, is somewhat more complicated. 

Figure [35] shows the basins of attraction, in 0(O)-v(O) space, of the asymmetric, 
mirror image, attractors pictured in Figs. [32] and [331 The basin of attraction of 
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Figure 35: The basins of attraction for the asymmetric, mirror image, attractors pictured in Figs. |32] 
and [33] Regions of 9(0)-v(0) space which lead to motion converging on the left-favouring attractor 
shown in Fig. \32\are coloured white: regions o/9(0)-v(0) space which lead to motion converging on 
the right-favouring attractor shown in Fig. \33\are coloured black. Data calculated numerically for 
Q = 1 .3, A = 1 .5, aj = 2/3, Nacc = 100, and 4> = 0. 



158 



4.7 Basins of Attraction 



4 THE CHAOTIC PENDULUM 




-1.00 -0.95 -0.90 -0.85 -0.80 -0.75 
6(0) / 77 

Figure 36: Detail of the basins of attraction for the asymmetric, mirror image, attractors pictured in 
Figs. \32\and\33\ Regions of 9(0)-v(0) space which lead to motion converging on the left-favouring 
attractor shown in Fig. \32\are coloured white: regions of 9(0)-v(0) space which lead to motion 
converging on the right-favouring attractor shown in Fig. [33] are coloured black. Data calculated 
numerically for Q = 1 3, A = 1 .5, w = 2/3, Nacc = 100, and cj) = 0. 

the left-favoring attractor shown in Fig. [32] is coloured black, whereas the basin 
of attraction of the right-favoring attractor shown in Fig. [33] is coloured white. It 
can be seen that the two basins form a complicated interlocking pattern. Since 
we can identify the angles n and — n, the right-hand edge of the pattern connects 
smoothly with its left-hand edge. In fact, we can think of the pattern as existing 
on the surface of a cylinder. 

Suppose that we take a diagonal from the bottom left-hand corner of Fig. [35] to 
its top right-hand corner. This diagonal is intersected by a number of black bands 
of varying thickness. Observe that the two narrowest bands (i.e., the fourth band 
from the bottom left-hand corner and the second band from the upper right- 
hand corner) both exhibit structure which is not very well resolved in the present 
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Figure 37: Detail of the basins of attraction for the asymmetric, mirror image, attractors pictured in 
Figs. 1221 and |33] Regions of 9(0)-v(0) space which lead to motion converging on the left-favouring 
attractor shown in Fig. \32\are coloured white: regions of 9(0)-v(0) space which lead to motion 
converging on the right-favouring attractor shown in Fig. |33] are coloured black. Data calculated 
numerically for Q = 1 3, A = 1 .5, w = 2/3, Nacc = 100, and cj) = 0. 



picture. 

Figure [36] is a blow-up of a region close to the lower left-hand corner of Fig. [351 
It can be seen that the unresolved band in the latter figure (i.e., the second and 
third bands from the right-hand side in the former figure) actually consists of 
a closely spaced pair of bands. Note, however, that the narrower of these two 
bands exhibits structure which is not very well resolved in the present picture. 



Figure [37] is a blow-up of a region of Fig. [36]. It can be seen that the unresolved 



band in the latter figure (i.e., the first and second bands from the left-hand side 
in the former figure) actually consists of a closely spaced pair of bands. Note, 
however, that the broader of these two bands exhibits structure which is not very 
well resolved in the present picture. 
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Figure 38: Detail of the basins of attraction for the asymmetric, mirror image, attractors pictured in 
Figs. 1221 and [33] Regions of 9(0)-v(0) space which lead to motion converging on the left-favouring 
attractor shown in Fig. \32\are coloured white: regions of 9(0)-v(0) space which lead to motion 
converging on the right-favouring attractor shown in Fig. [33] are coloured black. Data calculated 
numerically for Q = 1 3, A = 1 .5, w = 2/3, Nacc = 10Q, and cj) = 0. 
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Figure [38] is a blow-up of a region of Fig. [37]. It can be seen that the unresolved 
band in the latter figure (i.e., the first, second, and third bands from the right- 
hand side in the former figure) actually consists of a closely spaced triplet of 
bands. Note, however, that the narrowest of these bands exhibits structure which 
is not very well resolved in the present picture. 

It should be clear, by this stage, that no matter how closely we look at Fig. [35] 
we are going to find structure which we cannot resolve. In other words, the 
separatrix between the two basins of attraction shown in this figure is a curve 
which exhibits structure at all scales. Mathematicians have a special term for 
such a curve — they call it a fractal^ 

Many people think of fractals as mathematical toys whose principal use is the 
generation of pretty pictures. However, it turns out that there is a close con- 
nection between fractals and the dynamics of non-linear systems — particularly 
systems which exhibit chaotic dynamics. We have just seen an example in which 
the boundary between the basins of attraction of two co-existing attractors in 
phase-space is a fractal curve. This turns out to be a fairly general result: i.e., 
when multiple attractors exist in phase-space the separatrix between their var- 
ious basins of attraction is invariably fractal. What is this telling us about the 
nature of non-linear dynamics? Well, returning to Fig. [351 we can see that in the 
region of phase-space in which the fractal behaviour of the separatrix manifests 
itself most strongly (i.e., the region where the light and dark bands fragment) the 
system exhibits abnormal sensitivity to initial conditions. In other words, we only 
have to change the initial conditions slightly (i.e., so as to move from a dark to a 
light band, or vice versa) in order to significantly alter the time-asymptotic motion 
of the pendulum (i.e., to cause the system to converge to a left-favouring instead 
of a right-favouring attractor, or vice versa). Fractals and extreme sensitivity to 
initial conditions are themes which will reoccur in our investigation of non-linear 
dynamics. 
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Figure 39: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1 .5, cu = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, 
and c() = 0. 

4.8 Period-Doubling Bifurcations 



Let us now return to Fig. [301 Recall, that as the quality-factor Q is gradually 
increased, the time -asymptotic orbit of the pendulum through phase-space un- 
dergoes a sudden transition, at Q ~ 1.245, from a left-right symmetric, period- 1 
orbit to a left- favouring, period- 1 orbit. What happens if we continue to increase 
Q? Figure [39] is basically a continuation of Fig. [301 It can be seen that as Q is in- 
creased the left-favouring, period- 1 orbit gradually evolves until a critical value of 
Q, which is about 1 .348, is reached. When Q exceeds this critical value the nature 
of the orbit undergoes another sudden change: this time from a left-favouring, 
period- 1 orbit to a left-favouring, period-2 orbit. Obviously, the change is indi- 
cated by the forking of the curve in Fig. [391 This type of transition is termed a 
period-doubling bifurcation, since it involves a sudden doubling of the repetition 
period of the pendulum's time-asymptotic motion. 

We can represent period- 1 motion schematically as AAAAAA- • •, where A 
represents a pattern of motion which is repeated every period of the external 



27 B.B. Mandelbrot, The fractal geometry of nature, (W.H. Freeman, New York NY, 1982). 
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Figure 40: Equally spaced (in time) points on a time-asymptotic orbit in phase-space. Data calculated 
numerically for Q = 1 .36, A = 1 .5, w = 2/3, 9(0) = 0, v(0) = -3, and Nacc = 100. 



drive. Likewise, we can represent period-2 motion as ABABAB ■ ■ •, where A and 
B represent distinguishable patterns of motion which are repeated every alter- 
nate period of the external drive. A period-doubling bifurcation is represented: 
AAAAAA • • ■ — » ABABAB • • - . Clearly, all that happens in such a bifurcation is 
that the pendulum suddenly decides to do something slightly different in alter- 
nate periods of the external drive. 



Figure [40] shows the time-asymptotic phase-space orbit of the pendulum cal- 
culated for a value of Q somewhat higher than that required to trigger the above 
mentioned period-doubling bifurcation. It can be seen that the orbit is left- 
favouring (i.e., it spends the majority of its time on the left-hand side of the 
plot), and takes the form of a closed curve consisting of two interlocked loops in 
phase-space. Recall that for period- 1 orbits there was only a single closed loop in 
phase-space. Figure I4T1 shows the Poincare section of the orbit shown in Fig. |40]. 
The fact that the section consists of two points confirms that the orbit does indeed 
correspond to period-2 motion. 

A period-doubling bifurcation is an example of temporal symmetry breaking. 
The equations of motion of the pendulum are invariant under the transformation 
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Figure 41: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q = 1.36, 
A = 1 .5, o> = 2/3, 0(0) = 0, v(0) = 0, Nacc = 100, and cf> = 0. 



t — > t + t, where t is the period of the external drive. In the low amplitude (i.e., 
linear) limit the time -asymptotic motion of the pendulum always respects this 
symmetry. However, as we have just seen, in the non-linear regime it is possible 
to obtain solutions which spontaneously break this symmetry. Obviously, motion 
which repeats itself every two periods of the external drive is not as temporally 
symmetric as motion which repeats every period of the drive. 



Figure [42] is essentially a continuation of Fig [34]. Data is shown for two sets 
of initial conditions which lead to motions converging on left-favouring (lower 
branch) and right-favouring (upper branch) periodic attractors. We have already 
seen that the left-favouring periodic attractor undergoes a period-doubling bifur- 
cation at Q = 1 .348. It is clear from Fig. [42] that the right- favouring attractor un- 
dergoes a similar bifurcation at almost exactly the same Q -value. This is hardly 
surprising since, as has already been mentioned, for fixed physics parameters 
(i.e., Q, A, cu), the left- and right-favouring attractors are mirror-images of one 
another. 



165 



4.9 The Route to Chaos 



4 THE CHAOTIC PENDULUM 



2.0 



1.8 - 



1.6 



1.4 - 



1.2 - 



1.0 



1.30 




1.36 



Figure 42: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1.5, a> = 2/3, Nacc = 100, and 4> = 0. Data 
is shown for two sets of initial conditions: 9(0) = and v(0) = (lower branch); and 9(0) = and 
v(0) = —2 (upper branch). 

4.9 The Route to Chaos 



Let us return to Fig. [391 which tracks the evolution of a left- favouring periodic 
attractor as the quality-factor Q is gradually increased. Recall that when Q ex- 
ceeds a critical value, which is about 1 .348, then the attractor undergoes a period- 
doubling bifurcation which converts it from a period- 1 to a period-2 attractor. 
This bifurcation is indicated by the forking of the curve in Fig. [39]. Let us now 
investigate what happens as we continue to increase Q. Fig. [43] is basically a 
continuation of Fig. [39l It can be seen that, as Q is gradually increased, the at- 
tractor undergoes a period-doubling bifurcation at Q = 1 .348, as before, but then 
undergoes a second period-doubling bifurcation (indicated by the second forking 
of the curves) at Q ~ 1 .370, and a third bifurcation at Q ~ 1 .375. Obviously, the 
second bifurcation converts a period-2 attractor into a period-4 attractor (hence, 
two curves split apart to give four curves). Likewise, the third bifurcation con- 
verts a period-4 attractor into a period-8 attractor (hence, four curves split into 
eight curves) . Shortly after the third bifurcation, the various curves in the figure 
seem to expand explosively and merge together to produce an area of almost 
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Figure 43: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1 .5, cu = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, 
and c() = 0. 

solid black. As we shall see, this behaviour is indicative of the onset of chaos. 

Figure [44] is a blow-up of Fig. [43], showing more details of the onset of chaos. 
The period-4 to period-8 bifurcation can be seen quite clearly. However, we can 
also see a period-8 to period-16 bifurcation, at Q ~ 1.3755. Finally, if we look 
carefully, we can see a hint of a period-16 to period-32 bifurcation, just before 
the start of the solid black region. Figures [43] and [44] seem to suggest that the 
onset of chaos is triggered by an infinite series of period-doubling bifurcations. 

Table [2] gives some details of the sequence of period-doubling bifurcations 



shown in Figs. [43] and [44]. Let us introduce a bifurcation index n: the period- 1 to 
period-2 bifurcation corresponds to n = 1 ; the period-2 to period-4 bifurcation 
corresponds to n = 2; and so on. Let Q n be the critical value of the quality- 
factor Q above which the nth bifurcation is triggered. Table [2] shows the Q 
determined from Figs. [431 and 1441 for n = 1 to 5. Also shown is the ratio 



F n = Q " 1 n Qn 2 (4.27) 

— -1 
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Figure 44: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1 .5, cv = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, 
and c() = 0. 
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Bifurcation n Q n Q n — Qn-i Fn 

period- l^period-2 1 1 .34870 : ~- 

period-2^period-4 2 1.37003 0.02133 

period-4^period-8 3 1.37458 0.00455 4.69 ± 0.01 

period-8^period-16 4 1.37555 0.00097 4.69 ± 0.04 

period- 16^period-32 5 1.37575 0.00020 4.9 ±0.20 



Table 2: The period-doubling cascade. 



for n = 3 to 5. It can be seen that Tab. [2] offers reasonably convincing evidence 
that this ratio takes the constant value F = 4.69. It follows that we can esti- 
mate the critical Q -value required to trigger the nth bifurcation via the following 
formula: 

n-2 1 

Qn=Ql+(Q2-Q 1 )^ ?j , (4.28) 

j=0 

for n > 1 . Note that the distance (in Q) between bifurcations decreases rapidly 
as n increases. In fact, the above formula predicts an accumulation of period- 
doubling bifurcations at Q = Qoo, where 

OO , y 

Qoo = Qi + (Q 2 - Qi) £_ - = Qi + (Q 2 " Qi) -— r = 13758. (4.29) 

1=0 

Note that our calculated accumulation point corresponds almost exactly to the 
onset of the solid black region in Fig. [441 By the time that Q exceeds Q^, we 
expect the attractor to have been converted into a period-infinity attractor via an 
infinite series of period-doubling bifurcations. A period-infinity attractor is one 
whose corresponding motion never repeats itself, no matter how long we wait. 
In dynamics, such bounded aperiodic motion is generally referred to as chaos. 
Hence, a period-infinity attractor is sometimes called a chaotic attractor. Now, 
period-n motion is represented by n separate curves in Fig. [441 It is, therefore, 
not surprising that chaos (i.e., period-infinity motion) is represented by an infinite 
number of curves which merge together to form a region of solid black. 

Let us examine the onset of chaos in a little more detail. Figures 145H481 show 
details of the pendulum's time -asymptotic motion at various stages on the period- 
doubling cascade discussed above. Figure [45] shows period-4 motion: note that 



the Poincare section consists of four points, and the associated sequence of net 
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Figure 45: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q = 
1.372, A = 1.5, co = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, and cj) = 0. Also, shown is the net 
rotation per period, AQ/2n, calculated at the Poincare phase — 0. 



rotations per period of the pendulum repeats itself every four periods. Figure [46 



shows period-8 motion: now the Poincare section consists of eight points, and the 
rotation sequence repeats itself every eight periods. Figure [47] shows period- 16 
motion: as expected, the Poincare section consists of sixteen points, and the rota- 
tion sequence repeats itself every sixteen periods. Finally, Fig. [48] shows chaotic 



motion. Note that the Poincare section now consists of a set of four continuous 
line segments, which are, presumably, made up of an infinite number of points 
(corresponding to the infinite period of chaotic motion). Note, also, that the 
associated sequence of net rotations per period shows no obvious sign of ever re- 
peating itself. In fact, this sequence looks rather like one of the previously shown 
periodic sequences with the addition of a small random component. The genera- 



tion of apparently random motion from equations of motion, such as Eqs. (14.101) 
and (14.111) , which contain no overtly random elements is one of the most surpris- 
ing features of non-linear dynamics. 

Many non-linear dynamical systems, found in nature, exhibit a transition from 
periodic to chaotic motion as some control parameter is varied. Now, there are 
various known mechanisms by which chaotic motion can arise from periodic mo- 
tion. However, a transition to chaos via an infinite series of period-doubling 
bifurcations, as illustrated above, is certainly amongst the most commonly oc- 
curring of these mechanisms. Around 1975, the physicist Mitchell Feigenbaum 
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Figure 46: The Poincare section of a time- asymptotic orbit. Data calculated numerically for Q = 
1.375, A = 1.5, w = 2 A 9(0) = 0, v(0) = 0, Nacc = 100, and <J> = 0. Ako, s/iown is the net 
rotation per period, AQ/2n, calculated at the Poincare phase = 0. 




Figure 47: The Poincare section of a time- asymptotic orbit. Data calculated numerically for Q = 
1.3757, A = 1.5, w = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, and $ = 0. Ako, shown is the net 
rotation per period, AQ/2n, calculated at the Poincare phase = 0. 
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Figure 48: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q = 
1.376, A = 1.5, co = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, and cb = 0. A/so, s/iown is the net 
rotation per period, AQ/2n, calculated at the Poincare phase (b — 0. 



was investigating a simple mathematical model, known as the logistic map, which 
exhibits a transition to chaos, via a sequence of period-doubling bifurcations, as 
a control parameter r is increased. Let r n be the value of r at which the first 
2 n -period cycle appears. Feigenbaum noticed that the ratio 



r n -i - r n - 2 



TL 



r n -i 



(4.30) 



converges rapidly to a constant value, F = 4.669 . . ., as n increases. Feigenbaum 
was able to demonstrate that this value of F is common to a wide range of differ- 
ent mathematic models which exhibit transitions to chaos via period-doubling bi- 
furcations !l3 Feigenbaum went on to argue that the Feigenbaum ratio, F n , should 
converge to the value 4.669 ... in any dynamical system exhibiting a transition 
to chaos via period-doubling bifurcations @ This amazing prediction has been 
verified experimentally in a number of quite different physical systems]^ Note 
that our best estimate of the Feigenbaum ratio (see Tab. |2]) is 4.69 ± 0.01 , in good 
agreement with Feigenbaum's prediction. 

The existence of a universal ratio characterizing the transition to chaos via 
period-doubling bifurcations is one of many pieces of evidence indicating that 



28 M.J. Feigenbaum, Quantitative universality for a class of nonlinear transformations, J. Stat. Phys. 19, 25 (1978). 
29 M.J. Feigenbaum, The universal metric properties of nonlinear transformations, J. Stat. Phys. 21, 69 (1979). 

30t 



3 P. Citanovic, Universality in chaos, (Adam Hilger, Bristol UK, 1989). 
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chaos is a universal phenomenon (i.e., the onset and nature of chaotic motion 
in different dynamical systems has many common features). This observation 
encourages us to believe that in studying the chaotic motion of a damped, pe- 
riodically driven, pendulum we are learning lessons which can be applied to a 
wide range of non-linear dynamical systems. 

4.10 Sensitivity to Initial Conditions 

Suppose that we launch our pendulum and then wait until its motion has con- 
verged onto a particular attractor. The subsequent motion can be visualized as 
a trajectory 0o(t), vo(t) through phase-space. Suppose that we somehow perturb 
the pendulum, at time t = to, such that its position in phase-space is instanta- 
neously changed from 0o(to), vo(to) to 0o(to) + oQo, v (to) + 6v . The subsequent 
motion can be visualized as a second trajectory 0i (t), vi (t) through phase-space. 
What is the relationship between the original trajectory 0o(t), vo(t) and the per- 
turbed trajectory 0i(t), vi(t)? In other words, does the phase-space separation 
between the two trajectories, whose components are 



grow in time, decay in time, or stay more or less the same? What we are really 
investigating is how sensitive the time-asymptotic motion of the pendulum is to 
initial conditions. 

According to the linear analysis of Sect. 14.2 , 



60 (At) 
6v(At) 



0i(t o + At) -0 o (t o + At) 
Vi(t + At) -v (to + At) 



(4.31) 



(4.32) 



60 (At) 



69 cos(tu*At) e ~ At/2Q 




(4.33) 



6v(At) 



6v cos(o>*At) e ~ At/2Q 




(4.34) 
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Figure 49: The v-component of the separation between two neighbouring phase-space trajectories 
(one of which lies on an attractor) plotted against normalized time. Data calculated numerically for 
Q = 1 .372, A = 1 .5, cu = 2/3, 9(0) = 0, v(0) = 0, and Nacc = 100. The separation between the two 
trajectories is initialized to 8d = 5v = 10~ 6 at At = 0. 

assuming sin(cu*to) = 0. It is clear that in the linear regime, at least, the pen- 
dulum's time-asymptotic motion is not particularly sensitive to initial conditions. 
In fact, if we move the pendulum's phase-space trajectory slightly off the linear 
attractor, as described above, then the perturbed trajectory decays back to the 
attractor exponentially in time. In other words, if we wait long enough then the 
perturbed and unperturbed motions of the pendulum become effectively indistin- 
guishable. Let us now investigate whether this insensitivity to initial conditions 
carries over into the non-linear regime. 

Figures [4914521 show the results of the experiment described above, in which 
the pendulum's phase-space trajectory is moved slightly off an attractor and the 
phase-space separation between the perturbed and unperturbed trajectories is 
then monitored as a function of time, at various stages on the period-doubling 
cascade discussed in the previous section. To be more exact, the figures show 
the logarithm of the absolute magnitude of the v-component of the phase-space 
separation between the perturbed and unperturbed trajectories as a function of 
normalized time. 
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Figure 50: The v-component of the separation between two neighbouring phase-space trajectories 
(one of which lies on an attractor) plotted against normalized time. Data calculated numerically for 
Q = 1 .375, A = 1 .5, oj = 2/3, 9(0) = 0, v(0) = 0, and Nacc = 100. The separation between the two 
trajectories is initialized to 59 = 6v = 10~ 6 at At = 0. 
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Figure 51: The v-component of the separation between two neighbouring phase-space trajectories 
(one of which lies on an attractor) plotted against normalized time. Data calculated numerically for 
Q = 1.3757, A = 1.5, oj = 2/3, 9(0) = 0, v(0) = 0, and Nacc = 100. The separation between the 
two trajectories is initialized to 59 = 6v = 10~ 6 at At = 0. 
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Figure 52: The v-component of the separation between two neighbouring phase-space trajectories 
(one of which lies on an attractor) plotted against normalized time. Data calculated numerically for 
Q = 1 .376, A = 1 .5, cu = 2/3, 0(0) = 0, v(0) = 0, and Nacc = 100. The separation between the two 
trajectories is initialized to 8d = 5v = 10~ 6 at At = 0. 

Figure [49] shows the time evolution of the v-component of the phase-space 
separation, 5v, between two neighbouring trajectories, one of which is the period- 
4 attractor illustrated in Fig. [451 It can be seen that 6v decays rapidly in time. In 
fact, the graph of log(|6v|) versus At can be plausibly represented as a straight-line 
of negative gradient A. In other words, 

|6v(At)| ~ Sv e AA \ (4.35) 

where the quantity A is known as the Liapunov exponent. Clearly, in this case, A 
measures the strength of the exponential convergence of the two trajectories in 
phase-space. Of course, the graph of log(|6v|) versus At is not exactly a straight- 
line. There are deviations due to the fact that 6v oscillates, as well as decays, in 
time. There are also deviations because the strength of the exponential conver- 
gence between the two trajectories varies along the attractor. 

The above definition of the Liapunov exponent is rather inexact, for two main 
reasons. In the first place, the strength of the exponential convergence/divergence 
between two neighbouring trajectories in phase-space, one of which is an attrac- 
tor, generally varies along the attractor. Hence, we should really take formula 
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(14.351) and somehow average it over the attractor, in order to obtain a more 
unambiguous definition of A. In the second place, since the dynamical system 
under investigation is a second-order system, it actually possesses two different 
Liapunov exponents. Consider the evolution of an infinitesimal circle of per- 
turbed initial conditions, centred on a point in phase-space lying on an attrac- 
tor. During its evolution, the circle will become distorted into an infinitesimal 
ellipse. Let 6 k , where k = 1,2, denote the phase-space length of the kth princi- 
pal axis of the ellipse. The two Liapunov exponents, Ai and A2, are defined via 
6ic(At) ~ 6k(0) exp(A k At). However, for large At the diameter of the ellipse is 
effectively controlled by the Liapunov exponent with the most positive real part. 
Hence, when we refer to the Liapunov exponent, A, what we generally mean is 
the Liapunov exponent with the most positive real part. 

Figure [50] shows the time evolution of the v-component of the phase-space 
separation, 6v, between two neighbouring trajectories, one of which is the period- 
8 attractor illustrated in Fig. [461 It can be seen that 6v decays in time, though not 
as rapidly as in Fig. [491 Another way of saying this is that the Liapunov exponent 



of the periodic attractor shown in Fig. [46] is negative (i.e., it has a negative real 



part), though not as negative as that of the periodic attractor shown in Fig. [451 

Figure [5H shows the time evolution of the v-component of the phase-space 
separation, 6v, between two neighbouring trajectories, one of which is the period- 
16 attractor illustrated in Fig. [471 It can be seen that 6v decays weakly in time. 
In other words, the Liapunov exponent of the periodic attractor shown in Fig. [47 
is small and negative. 

Finally, Fig. [52] shows the time evolution of the v-component of the phase- 
space separation, 6v, between two neighbouring trajectories, one of which is 
the chaotic attractor illustrated in Fig. [481 It can be seen that 6v increases in 
time. In other words, the Liapunov exponent of the chaotic attractor shown in 
Fig. [48] is positive. Further investigation reveals that, as the control parameter Q 
is gradually increased, the Liapunov exponent changes sign and becomes positive 
at exactly the same point that chaos ensues in Fig [441 

The above discussion strongly suggests that periodic attractors are character- 
ized by negative Liapunov exponents, whereas chaotic attractors are character- 
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ized by positive exponents. But, how can an attractor have a positive Liapunov ex- 
ponent? Surely, a positive exponent necessarily implies that neighbouring phase- 
space trajectories diverge from the attractor (and, hence, that the attractor is not a 
true attractor)? It turns out that this is not the case. The chaotic attractor shown 



in Fig. [48] is a true attractor, in the sense that neighbouring trajectories rapidly 
converge onto it — i.e., after a few periods of the external drive their Poincare sec- 
tions plot out the same four-line segment shown in Fig. [48} Thus, the exponential 



divergence of neighbouring trajectories, characteristic of chaotic attractors, takes 
place within the attractor itself. Obviously, this exponential divergence must come 
to an end when the phase-space separation of the trajectories becomes compara- 
ble to the extent of the attractor. 

A dynamical system characterized by a positive Liapunov exponent, A, has a 
time horizon beyond which regular deterministic prediction breaks down. Sup- 
pose that we measure the initial conditions of an experimental system very ac- 
curately. Obviously, no measurement is perfect: there is always some error 6o 
between our estimate and the true initial state. After a time t, the discrepancy 
grows to 6(t) ~ 6o exp(A t). Let a be a measure of our tolerance: i.e., a prediction 
within a of the true state is considered acceptable. It follows that our prediction 
becomes unacceptable when 6 ^> a, which occurs when 

t> tH-ilng). (4.36) 

Note the logarithmic dependence on 5o. This ensures that, in practice, no matter 
how hard we work to reduce our initial measurement error, we cannot predict 
the behaviour of the system for longer than a few multiples of 1 /A. 

It follows, from the above discussion, that chaotic attractors are associated 
with motion which is essentially unpredictable. In other words, if we attempt to 
integrate the equations of motion of a chaotic system then even the slightest error 
made in the initial conditions will be amplified exponentially over time and will 
rapidly destroy the accuracy of our prediction. Eventually, all that we will be able 
to say is that the motion lies somewhere on the chaotic attractor in phase-space, 
but exactly where it lies on the attractor at any given time will be unknown to us. 

The hyper-sensitivity of chaotic systems to initial conditions is sometimes called 
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the butterfly effect. The idea is that a butterfly flapping its wings in a South 
American rain-forest could, in principle, affect the weather in Texas (since the 
atmosphere exhibits chaotic dynamics). This idea was first publicized by the 
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the con- 
Lorenz 



meteorologist Edward Lorenz, who constructed a very crude model o 
vection of the atmosphere when it is heated from below by the ground, 
discovered, much to his surprise, that his model atmosphere exhibited chaotic 
motion — which, at that time, was virtually unknown to physics. In fact, Lorenz 
was essentially the first scientist to fully understand the nature and ramifications 
of chaotic motion in physical systems. In particular, Lorenz realized that the 
chaotic dynamics of the atmosphere spells the doom of long-term weather fore- 
casting: the best one can hope to achieve is to predict the weather a few days in 
advance (1 /A for the atmosphere is of order a few days). 



4. 1 1 The Definition of Chaos 

There is no universally agreed definition of chaos. However, most people would 
accept the following working definition: 

Chaos is aperiodic time-asymptotic behaviour in a deterministic system 
which exhibits sensitive dependence on initial conditions. 

This definition contains three main elements: 

1. Aperiodic time-asymptotic behaviour — this implies the existence of phase-space 
trajectories which do not settle down to fixed points or periodic orbits. For 
practical reasons, we insist that these trajectories are not too rare. We also 
require the trajectories to be bounded: i.e., they should not go off to infinity. 

2. Deterministic — this implies that the equations of motion of the system possess 
no random inputs. In other words, the irregular behaviour of the system 
arises from non-linear dynamics and not from noisy driving forces. 

31 E. Lorenz, Deterministic nonperiodicflow, J. Atmospheric Science 20, 130 (1963). 
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Figure 53: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1.5,, cu = 2/3, 9(0) = 0, v(0) = —0.75, 
Nacc = 100, and $ = 0. 

3. Sensitive dependence on initial conditions — this implies that nearby trajectories 
in phase-space separate exponentially fast in time: i.e., the system has a 
positive Liapunov exponent. 

4.12 Periodic Windows 

Let us return to Fig. [441 Recall, that this figure shows the onset of chaos, via 
a cascade of period-doubling bifurcations, as the quality-factor Q is gradually 
increased. Figure [53] is essentially a continuation of Fig. [441 which shows the full 
extent of the chaotic region (in Q-v space). It can be seen that the chaotic region 
ends abruptly when Q exceeds a critical value, which is about 1 .4215. Beyond this 
critical value, the time-asymptotic motion appears to revert to period- 1 motion 
(i.e., the solid black region collapses to a single curve). It can also be seen that the 
chaotic region contains many narrow windows in which chaos reverts to periodic 
motion (i.e., the solid black region collapses to n curves, where n is the period of 
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Figure 54: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1.5,, cu = 2/3, 9(0) = 0, v(0) = —0.75, 
Nacc = 100, and $ = 0. 

the motion) for a short interval in Q. The four widest windows are indicated on 
the figure. 

Figure [54] is a blow-up of the period-3 window shown in Fig. [531 It can be seen 
that the window appears "out of the blue" as Q is gradually increased. However, 
it can also be seen that, as Q is further increased, the window breaks down, 
and eventually disappears, due to the action of a cascade of period-doubling 
bifurcations. The same basic mechanism operates here as in the original period- 



doubling cascade, discussed in Sect. 14.91 except that now the orbits are of period 



3 • 2 n , instead of 2 • 2 n . Note that all of the other periodic windows seen in Fig. [53] 
break down in an analogous manner, as Q is increased. 

We now understand how periodic windows break down. But, how do they 
appear in the first place? Figures [551457] show details of the pendulum's time- 



asymptotic motion calculated just before the appearance of the period-3 window 
(shown in Fig. 1541) , just at the appearance of the window, and just after the ap- 
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Figure 55: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q = 
1.387976, A = 1 .5, cu = 2/3, 9(0) = 0, v(0) = -0.75, Nacc = 100, and $ = 0. Aiso, shown is the 
net rotation per period, Ad/2n, calculated at the Poincare phase — 0. 
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Figure 56: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q = 
1.387977, A = 1.5, w = 2/3, 9(0) = 0, v(0) = -0.75, Nacc = 100, and $ = 0. Ako, shown is the 
net rotation per period, AQ/2n, calculated at the Poincare phase c|) = 0. 
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Figure 57: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q = 
1.387978, A = 1.5, w = 2/3, 9(0) = 0, v(0) = -0.75, Nacc = 100, and $ = 0. Ako, shown is the 
net rotation per period, Ad/2n, calculated at the Poincare phase = 0. 



pearance of the window, respectively. It can be seen, from Fig. [55], that just before 
the appearance of the window the attractor is chaotic (i.e., its Poincare section 
consists of a line, rather than a discrete set of points), and the time-asymptotic 
motion of the pendulum consists of intervals of period-3 motion interspersed with 
intervals of chaotic motion. Figure [56] shows that just at the appearance of the 
window the attractor loses much of its chaotic nature (i.e., its Poincare section 
breaks up into a series of points), and the chaotic intervals become shorter and 
much less frequent. Finally, Fig. [57] shows that just after the appearance of the 
window the attractor collapses to a period-3 attractor, and the chaotic intervals 
cease altogether. All of the other periodic windows seen in Fig. [53] appear in an 
analogous manner to that just described. 

According to the above discussion, the typical time -asymptotic motion seen 
just prior to the appearance of a period-n window consists of intervals of period- 
n motion interspersed with intervals of chaotic motion. This type of behaviour 
is called intermittency, and is observed in a wide variety of non-linear systems. 
As we move away from the window, in parameter space, the intervals of periodic 
motion become gradually shorter and more infrequent. Eventually, they cease 
altogether. Likewise, as we move towards the window, the intervals of periodic 
motion become gradually longer and more frequent. Eventually, the whole mo- 
tion becomes periodic. 
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In 1973, Metropolis and co-workers investigated a class of simple mathemati- 
cal models which all exhibit a transition to chaos, via a cascade of period-doubling 



bifurcations, as some control parameter r is increased o They were able to 
demonstrate that, for these maps, the order in which stable periodic orbits occur 
as r is increased is fixed. That is, stable periodic attractors always occur in the same 
sequence as r is varied. This sequence is called the universal or V -sequence. It is 
possible to make a fairly convincing argument that any physical system which ex- 
hibits a transition to chaos via a sequence of period-doubling bifurcations should 
also exhibit the U-sequence of stable periodic attractors. Up to period-6, the U- 
sequence is 

1 , 2, 2 x 2, 6, 5, 3, 2 x 3, 5, 6, 4, 6, 5, 6. 

The beginning of this sequence is familiar: periods 1, 2, 2 x 2 are the first stages 
of the period-doubling cascade. (The later period-doublings give rise to periods 
greater than 6, and so are omitted here). The next periods, 6,5,3 correspond to 
the first three periodic windows shown in Fig. [531 Period 2 x 3 is the first compo- 
nent of the period-doubling cascade which breaks up the period-3 window. The 
next period, 5, corresponds to the last periodic window shown in Fig. [531 The 
remaining periods, 6, 4, 6, 5, 6, correspond to tiny periodic windows, which, in 
practice, are virtually impossible to observe. It follows that our driven pendulum 
system exhibits the U-sequence of stable periodic orbits fairly convincingly. This 
sequence has also been observed experimentally in other, quite different, dynam- 



ical systems O The existence of a universal sequence of stable periodic orbits 
in dynamical systems which exhibit a transition to chaos via a cascade of period- 
doubling bifurcations is another indication that chaos is a universal phenomenon. 



4.13 Further Investigation 



Figure [58] shows the complete bifurcation diagram for the damped, periodically 
driven, pendulum (with A = 1.5 and cu = 2/3). It can be seen that the chaotic 



32 N. Metropolis, M.L. Stein, and P.R. Stein, On finite limit sets for transformations on the unit interval, J. Combin. 
Theor. 15, 25 (1973). 

33 R.H. Simoyi, A. Wolf, and H.L. Swinney, One-dimensional dynamics in a multi-component chemical reaction, Phys. 
Rev. Lett. 49, 245 (1982). 
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Figure 58: The v-coordinate of the Poincare section of a time-asymptotic orbit plotted against the 
quality-factor Q. Data calculated numerically for A = 1 .5, cv = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, 
and c() = 0. 
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Figure 59: Equally spaced (in time) points on a time-asymptotic orbit in phase-space. Data calculated 
numerically for Q = 1 .5, A = 1 .5, cv = 2/3, 8(0) = 0, v(0) = 0, and Nacc = 100. Also shown is the 
time-asymptotic orbit calculated for the modified initial conditions 9(0) = 0, and v(0) = —1. 

region investigated in the previous section is, in fact, the first, and least extensive, 
of three different chaotic regions. 

The interval between the first and second chaotic regions is occupied by the 
period- 1 orbits shown in Fig. [591 Note that these orbits differ somewhat from 
previously encountered period- 1 orbits, because the pendulum executes a com- 
plete rotation (either to the left or to the right) every period of the external drive. 
Now, an n, I periodic orbit is defined such that 

9(t + TiT) = 9(t) +2n\ 

for all t (after the transients have died away). It follows that all of the periodic 
orbits which we encountered in previous sections were 1 = orbits: i.e., their as- 
sociated motions did not involve a net rotation of the pendulum. The orbits show 
in Fig. [59] are n = 1 , 1 = — 1 and n = 1 , 1 = +1 orbits, respectively. The existence 
of periodic orbits in which the pendulum undergoes a net rotation, either to the 
left or to the right, is another example of spatial symmetry breaking — there is 
nothing in the pendulum's equations of motion which distinguishes between the 
two possible directions of rotation. 



Figure [60] shows the Poincare section of a typical attractor in the second chaotic 
region shown in Fig. [58]. It can be seen that this attractor is far more convoluted 
and extensive than the simple 4-line chaotic attractor pictured in Fig. [481 In fact, 
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Figure 60: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q =2.13, 
A = 1 .5, o> = 2/3, 0(0) = 0, v(0) = 0, Nacc = 100, and cf> = 0. 



the attractor shown in Fig. [60] is clearly a fractal curve. It turns out that virtually 
all chaotic attr actors exhibit fractal structure. 



The interval between the second and third chaotic regions shown in Fig. [58] is 
occupied by n = 3, I = periodic orbits. Figure [61] shows the Poincare section 
of a typical attractor in the third chaotic region. It can be seen that this attractor 
is even more overtly fractal in nature than that pictured in the previous figure. 
Note that the fractal nature of chaotic attractors is closely associated with some 
of their unusual properties. Trajectories on a chaotic attractor remain confined 
to a bounded region of phase-space, and yet they separate from their neighbours 
exponentially fast (at least, initially). How can trajectories diverge endlessly and 
still stay bounded? The basic mechanism is described below. If we imagine a 
blob of initial conditions in phase-space, then these undergo a series of repeated 
stretching and folding episodes, as the chaotic motion unfolds. The stretching is 
what gives rise to the divergence of neighbouring trajectories. The folding is what 
ensures that the trajectories remain bounded. The net result is a phase-space 
structure which looks a bit like filo dough — in other words, a fractal structure. 
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Figure 61: The Poincare section of a time-asymptotic orbit. Data calculated numerically for Q = 3.9, 
A = 1 .5, uo = 2/3, 9(0) = 0, v(0) = 0, Nacc = 100, and $ = 0. 
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5 Poisson's Equation 
5.1 Introduction 

In this section, we shall discuss some simple numerical techniques for solving 
Poisson's equation: 

V 2 u(r) =v(r). (5.1) 

Here, u(r) is usually some sort of potential, whereas v(r) is a source term. The 
solution to the above equation is generally required in some simply-connected 
volume V bounded by a closed surface S. There are two main types of boundary 
conditions to Poisson's equation. In so-called Dirichlet boundary conditions, the 
potential u is specified on the bounding surface S. In so-called Neumann bound- 
ary conditions, the normal gradient of the potential Vu • dS is specified on the 
bounding surface. 

Poisson's equation is of particular importance in electrostatics and Newtonian 
gravity. In electrostatics, we can write the electric field E in terms of an electric 
potential cj): 

E = -Vc|). (5.2) 
The potential itself satisfies Poisson's equation: 

V 2 c() = -— , (5.3) 

eo 

where p(r) is the charge density, and eo the permittivity of free-space. In New- 
tonian gravity, we can write the force f exerted on a unit test mass in terms of a 
gravitational potential 4>: 

f=-Vc|). (5.4) 
The potential satisfies Poisson's equation: 

V 2 c() =4tt 2 G p, (5.5) 

where p(r) is the mass density, and G the universal gravitational constant. 
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5.2 1-D Problem with Dirichlet Boundary Conditions 

As a simple test case, let us consider the solution of Poisson's equation in one 
dimension. Suppose that 

dx2 =v(x), (5.6) 

for xi < x < Xh, subject to the Dirichlet boundary conditions u(xj = ui and 
u(x h ) = u h . 

As a first step, we divide the domain x\ < x < Xh into equal segments whose 
vertices are located at the grid-points 

xt = x L + — — - , (5.7) 

for i = 1 , N . The boundaries, x\ and Xh, correspond to i = and i = N + 1 , 
respectively. 

Next, we discretize the differential term d 2 u/dx 2 on the grid-points. The most 
straightforward discretization is 

d 2 u(xi) Ui_] -2u t + u i+1 2 

-&- = (&x? + 0(6X1 • (5 - 8) 

Here, 6x = (xh — xO/(N + 1), and u t = u(xi). This type of discretization is 
termed a second-order, central difference scheme. It is "second-order" because the 
truncation error is 0(6x) 2 , as can easily be demonstrated via Taylor expansion. Of 
course, an nth order scheme would have a truncation error which is 0(6x) n . It is 
a "central difference" scheme because it is symmetric about the central grid-point, 
xi. Our discretized version of Poisson's equation takes the form 

ut^ - 2u t + u i+ i = Vi (6x) 2 , (5.9) 

for i = 1 , N, where v t = v(xt). Furthermore, uo = ui and un+i = u h . 

It is helpful to regard the above set of discretized equations as a matrix equa- 
tion. Let u = (ui , U2, • • • , un) be the vector of the u- values, and let 

w = [v] (6x) 2 - u h v 2 (6x) 2 , v 3 (6x) 2 , • • • , v n _t (6x) 2 , v n (6x) 2 - u h ] (5.10) 
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be the vector of the source terms. The discretized equations can be written as: 



Mu = w. 



(5.11) 



The matrix M takes the form 



M 



V 



-2 
1 








1 

-2 
1 









1 

-2 
1 









1 

-2 
1 









1 

-2 
1 



o\ 





1 

"2/ 



(5.12) 



for N = 6. The generalization to other N values is fairly obvious. Matrix M 
is termed a tridiagonal matrix, since only those elements which lie on the three 
leading diagonals are non-zero. 



The formal solution to Eq. (15.111) is 



u = M 1 w, 



(5.13) 



where M 1 is the inverse matrix to M. Unfortunately the most efficient general 
purpose algorithm for inverting anNxN matrix — namely Gauss-Jordan elimina- 
tion with partial pivoting — requires 0(N 3 ) arithmetic operations. It is fairly clear 
that this is a disastrous scaling for finite-difference solutions of Poisson's equation. 
Every time we doubled the resolution (i.e., doubled the number of grid-points) 
the required cpu time would increase by a factor of about eight. Consequently 
adding a second dimension (which effectively requires the number of grid-points 
to be squared) would be prohibitively expensive in terms of cpu time. Fortunately 
there is a well-known trick for inverting an N x N tridiagonal matrix which only 
requires O(N) arithmetic operations. 

Consider a general N x N tridiagonal matrix equation Mu = w. Let a, b, 
and c be the vectors of the left, center and right diagonal elements of the matrix, 
respectively Note that a-\ and cm are undefined, and can be conveniently set to 
zero. Our matrix equation can now be written 



CliUi-T + biUi + CiU i+1 = w it 



(5.14) 
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for i = 1 , N . Let us search for a solution of the form 

u i+1 = XiU t + yi. (5.15) 
Substitution into Eq. (I5.14D yields 

ciiUi i + biUi + c t (xiUi + Vi) = Wt, (5.16) 
which can be rearranged to give 

CliUi_i Wi-CiUi 

iU = -r— . + tt^ • (5.17) 



However, if Eq. (15.151) is general then we can write Ui = Xj._i U| i + yi i . Com- 



parison with the previous equation yields 



and 



Xi_i=-— - , (5.18) 

bi + ch Xi 

Vi-i = t— . (5.19) 
bi + Ci x t 

We can now solve our tridiagonal matrix equation in two stages. In the first stage, 
we scan up the leading diagonal from i = N to 1 using Eqs. (15.181) and (I5.19D . 
Thus, 

xn i = — , , Un i = , , (5.20) 
b N b N 



since cn = 0. Furthermore, 



CLi+i w i+ i - c i+1 u i+1 rcon 

bi+i + Ct+i Xi+i b i+1 + c i+1 x i+1 



for i = N — 2, 1 . Finally, 



*o = 0, "yo = T— , (5.22) 

b] + ci X] 



since cii = 0. We have now defined all of the x\ and y\. In the second stage, we 
scan down the leading diagonal from i = to N — 1 using Eq. (15.151) . Thus, 

U] =y 0} (5.23) 
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since xo = 0, and 

u i+1 = xtUi + iji (5.24) 

for i = 1 , N — 1 . We have now inverted our tridiagonal matrix equation using 
O(N) arithmetic operations. 



Clearly, we can use the above algorithm to invert Eq. (15. IIP , with the source 



terms specified in Eq. (I5.10D . and the diagonals of matrix M given by a t = 1 for 



i = 2, N, plus bi = — 2 for i = 1 , N, and Ci = 1 for i = 1 , N — 1 . 



5.3 An Example Tridiagonal Matrix Solving Routine 

Listed below is an examp le trid iagonal matrix solving routine which utilizes the 
Blitz++ library (see Sect. [2 i 20J). 

// Tridiagonal . cpp 

// Function to invert tridiagonal matrix equation. 

// Left, centre, and right diagonal elements of matrix 

// stored in arrays a, b, c, respectively. 

// Right-hand side stored in array w. 

// Solution written to array u. 

// Matrix is NxN. Arrays a, b, c, w, u assumed to be of extent N+2, 
// with redundant and N+l elements. 

#include <blitz/array .h> 

using namespace blitz; 

void Tridiagonal (Array<double , 1> a, Array<double , 1> b, Array<double , 1> c, 
Array<double , 1> w, Array<double , 1>& u) 

{ 

// Find N. Declare local arrays, 
int N = a. extent (0) - 2; 
Array<double, 1> x(N) , y(N) ; 

// Scan up diagonal from i = N to 1 

x(N-l) = - a(N) / b(N) ; 

y(N-l) = w(N) / b(N); 

for (int i = N-2; i > 0; i — ) 
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{ 

x(i) = - a(i+l) / (b(i+l) + c(i+l) * x(i+l)); 

y(i) = (w(i+l) - c(i+l) * y(i+D) / (b(i+l) + c(i+l) * x(i+l)); 

} 

x(0) = 0.; 

y(0) = (w(l) - c(l) * y(l)) / (b(l) + c(l) * x(l)); 

// Scan down diagonal from i = to N-l 
u(l) = y(0); 

for (int i = 1; i < N; i++) 
u(i+l) = x(i) * u(i) + y(i) ; 



5.4 1-D problem with Mixed Boundary Conditions 

Previously, we solved Poisson's equation in one dimension subject to Dirichlet 
boundary conditions, which are the simplest conceivable boundary conditions. 
Let us now consider the following much more general set of boundary conditions: 

du(x) 

oci u(x) + (3 L — — = yt, (5.25) 
dx 

at x = X\, and 

du(x) 

cx h u(x) + p h — = y H , (5.26) 
dx 

at x = Xh- Here, oc\, etc., are known constants. The above boundary conditions 
are termed mixed, since they are a mixture of Dirichlet and Neumann boundary 
conditions. 



Using the previous notation, the discretized versions of Eq. (15.251) and (15.261) 
are: 

Ui — Uo , . 

cxlUo + Pl — = yi, (5.27) 

ox 

n U N + 1 — UN rc 00 . 

cx h u N+1 + (3 h = yh> (5.28) 

ox 

respectively. The above expressions can be rearranged to give 

u = . ^— , (5.29) 

oci bx - pi 
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Y h 6x+p K u N 
un+i = = — —5 — • (5.30) 

Ct h bx + ph 



Using Eqs. (15. 8D . (15.291) , and (15.301) . the problem can be reduced to a tridiag- 



onal matrix equation M u = w, where the left, center, and right diagonals of M 
possess the elements a\ — 1 for i = 2, N, with 

pl (5.31) 



ociSx - (V 
and bi = —2 for i = 2, N — 1 , plus 

b N = -2+ Ph _ , (5.32) 
oc h bx + Ph 

and Ci = 1 for i = 1 , N — 1 , respectively. The elements of the right-hand side are 

w^vAbx) 2 - l lbX a , (5.33) 

0L\ bx - pi 

with w t = v\ [bx] 1 for i = 2, N — 1 , and 

w N = v N (6x) 2 - yhb * . (5.34) 

oc h bx + (3h 

Our tridiagonal matrix equation can be inverted using the algorithm discussed 
previously. 



5.5 An Example 1-D Poisson Solving Routine 

Listed below is an example 1-d Poisson solving routine which utilizes t he pr evi- 
ously listed tridiagonal matrix solver and the Blitz++ library (see Sect. 12-2(3) • 



// PoissonlD . cpp 

// Function to solve Poisson' s equation in 1-d: 
// d"2 u / dx~2 = v for xl <= x <= xh 
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// alpha_l u + beta_l du/dx = gamma_l at x=xl 
// alpha_h u + beta_h du/dx = gamma_h at x=xh 
// Arrays u and v assumed to be of extent N+2. 
// Now, ith elements of arrays correspond to 
// x_i = xl + i * dx i=0,N+l 
// Here, dx = (xh - xl) / (N+l) is grid spacing. 
#include <blitz/array.h> 
using namespace blitz; 

void Tridiagonal (Array<double , 1> a, Array<double , 1> b, Array<double , 1> c, 
Array<double , 1> w, Array<double , 1>& u) ; 

void PoissonlD (Array<double , 1>& u, Array<double , 1> v, 

double alpha_l, double beta_l, double gamma_l, 
double alpha_h, double beta_h, double gamma_h, 
double dx) 

{ 

// Find N. Declare local arrays, 
int N = u. extent (0) - 2; 

Array<double,l> a(N+2) , b(N+2) , c(N+2), w(N+2) ; 

// Initialize tridiagonal matrix 
for (int i = 2; i <= N; i++) a(i) = 1.; 
for (int i = 1; i <= N; i++) b(i) = -2.; 
b(l) -= beta_l / (alpha_l * dx - beta_l) ; 
b(N) += beta_h / (alpha_h * dx + beta_h) ; 
for (int i = 1; i <= N-l; i++) c(i) = 1.; 

// Initialize right-hand side vector 
for (int i = 1; i <= N; i++) 

w(i) = v(i) * dx * dx; 
w(l) -= gamma_l * dx / (alpha_l * dx - beta_l) ; 
w(N) -= gamma_h * dx / (alpha_h * dx + beta_h) ; 

// Invert tridiagonal matrix equation 
Tridiagonal (a, b, c, w, u) ; 

// Calculate i=0 and i=N+l values 
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u(0) = (gamma_l * dx - beta_l * u(l)) / 

(alpha_l * dx - beta_l) ; 
u(N+l) = (gamma_h * dx + beta_h * u(N)) / 

(alpha_h * dx + beta_h) ; 

} 



5.6 An Example Solution ofPoisson's Equation in 1-D 

Let us now solve Poisson's equation in one dimension, with mixed boundary con- 
ditions, using the finite difference technique discussed above. We seek the solu- 
tion of 

dx 2 =V M> (5.35) 
in the region < x < 1, with v(x) = 1 — 2x 2 . The boundary conditions at xi = 



and Xh = 1 take the mixed form specified in Eqs. (15.251) and (15.261) . Of course, 
we can solve this problem analytically. In fact, 

x 2 x 4 

u(x) = g + hx + — - — , (5.36) 

L D 

where 

g = , (5.37) 

oci oc h + Oil Ph - Pi an 

h = Qi\ hh - {oth + Ph)/3] - Ti oc h ^ 3g 

oci oc h + ai (3 h - Pi an 

Figure [62] shows a comparison between the analytic and finite difference solutions 
for N = 100. It can be seen that the finite difference solution mirrors the analytic 
solution almost exactly. 



5.7 2-D problem with Dirichlet Boundary Conditions 

Let us consider the solution of Poisson's equation in two dimensions. Suppose 
that 

3 2 u(x,u) 3 2 u(x,-y) 

q x 2 + 9^2 = V ( X >^> (5 - 39) 
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Figure 62: Solution of Poisson's equation in one dimension with v = 1 — 2x 2 , oci = ], fi\ = — \, 
Yx = 1, cch = L ph = L and Th = 1- The dotted curve (obscured) shows the analytic solution, 
whereas the open triangles show the finite difference solution for N = 100. 

for x\ < x < Xh, and < y < L. By direct analogy with our previous method of so- 
lution in the 1-d case, we could discretize the above 2-d problem using a second- 
order, central difference scheme in both the x- and y -directions. Unfortunately, 
such a discretization scheme yields a set of equations which cannot be reduced 
to a simple tridiagonal matrix equation. In fact, all of the efficient numerical al- 
gorithms for solving this type of problem are iterative in nature. For instance, the 
Jacobi method, the Gauss-Seidel method, the successive over-relaxation method, 
and the multi-grid method@ Regrettably, unless such iteration methods are ex- 
tremely sophisticated (e.g., the multi-grid method), and, hence, beyond the scope 
of this course, they tend to converge very poorly. In the following, rather than 
discuss iterative methods which do not work very well, we shall instead discuss 
a non-iterative method which works effectively for a restricted set of problems. 
The method in question is termed a spectral method, since it involves expanding 
u and v as truncated Fourier series in the y -direction. 

Suppose that u(x,y) satisfies mixed boundary conditions in the x-direction: 

34 See Numerical recipes in C: the art of scientific computing, W.H. Press, S.A. Teukolsky, W.T. Vettering, and B.R. Flan- 
nery (Cambridge University Press, Cambridge, England, 1992), Sect. 19.5. 
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i.e., 

odu(x t y) + = Tl (u), (5.40) 

ox 

at x = xi, and 

, , . 9u(x,u) , , ,_ 

cx h u(x,uj + p h — =yh(y), (5.41) 

ox 

at x = Xh- Here, oc\, (3i, etc., are known constants, whereas yi, yn are known 
functions of y. Furthermore, suppose that u(x,y) satisfies the following simple 
Dirichlet boundary conditions in the y -direction: 

u(x,0) =u(x,L) = 0. (5.42) 

Note that, since u(x,y) is a potential, and, hence, probably undetermined to 
an arbitrary additive constant, the above boundary conditions are equivalent to 
demanding that u take the same constant value on both the upper and lower 
boundaries in the y -direction. 

Let us write u(x,y) as a Fourier series in the u-direction: 

oo 

u(x,y) = ^Uj(x) sin(j7ry/L). (5.43) 

j=0 

Note that the above expression for u automatically satisfies the boundary con- 
ditions in the u-direction. The sin(j ny/L) functions are orthogonal, and form a 
complete set, in the interval y = 0, L. In fact, 

2 r L 

- sin(j ny/L) sin(k7tu/L) dy = 6 jk . (5.44) 
L Jo 

Thus, we can write the source term as 

oo 

v(x,y) =^Vj(x) sin(j7ry/L), (5.45) 

j=0 



where 



V,(x) = 



L 



v(x,y) sin(j7tu/L) dy. (5.46) 



199 



5. 7 2-D problem with Dirichlet Boundary Conditions 



5 POISSON'S EQUATION 



Furthermore, the boundary conditions in the x-direction become 

dllj(x) 



at x = x\, and 



at x = Xh, where 



etc. 



ocilij(x) + |3 L 



a K Uj(x) + p h 



dx 

dUj(x) 
dx 



rij, 



Yi{y) sin(j7ru/L) dy, 



(5.47) 



(5.48) 



(5.49) 



Substituting Eqs. (I5.43D and (I5.45D into Eq. (15.39D , and equating the coeffi- 
cients of the sin(j ny/U) (since these functions are orthogonal), we obtain 



d 2 lL(x) 



j 2 7I 2 



U 1 (x)=V,(x), 



(5.50) 



dx 2 L 2 

for j = 0, oo. Now, we can discretize the problem in the y -direction by truncating 
our Fourier expansion: i.e., by only solving the above equations for j = 0, J, rather 
than j = 0,oo. This is essentially equivalent to discretization in the y -direction 
on the equally-spaced grid-points Uj = j L/J. The problem is discretized in the x- 
direction by dividing the domain into equal segments, according to Eq. (I5.7I) , and 
approximating d 2 /dx 2 via the second-order, central difference scheme specified 
in Eq. (I5.8D . Thus, we obtain 

U t _ 1(j - (2 + j 2 k 2 ) U i(j + Ut+i j = Vu (6x) 2 , 



(5.51) 



for i = 1,N and j = 0, J. Here, = Uj(Xi), Vy = Vj(xi), and k = nbx/L. The 
boundary conditions (15.471) and (15.481) discretize to give: 



(5.52) 
(5.53) 



oc\ bx — (3 1 
r hj 5x+ (3 h U NJ 
cxh6x+(3h 

for j = 0, J. Eqs. (15.511) . (15.521) . and (15.531) constitute a set of J + 1 uncoupled 
tridiagonal matrix equations (with one equation for each separate j value) . These 
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equations can be inverted, using the algorithm discussed in Sect. 15.41 to give the 
Ui j. Finally, the u(xi,Uj) values can be reconstructed from Eq. (15.431) . Hence, we 
have solved the problem. 



5.8 2-d Problem with Neumann Boundary Conditions 



Let us redo the above calculation, replacing the Dirichlet boundary conditions 
(15.421) with the following simple Neumann boundary conditions: 



0. 



3u(x,y=0) 3u(x,y = L) 
3y 3y 

In this case, we can express u(x,y) in the form 

00 

u(x,y) = ^Uj(x) cos(j7ry/L), 

i=o 



(5.54) 



(5.55) 



which automatically satisfies the boundary conditions in the y -direction. Like- 
wise, we can write the source term v(x,y) as 



v(x,y) = Y_ y jM cos(j7tu/L), 



where 



since 



V j W = L 
2 ^ L 



v(x,y) cos(j7iu/L) dy, 



cos(j7ty/L) cos(k7ry/L) dy =6^. 



Finally, the boundary conditions in the x-direction become 



at x = x\, and 



ociUj(x) + (3 L 



a K Uj(x) + p h 



dllj(x) 
dx 



dUj(x) 
dx 



(5.56) 



(5.57) 



(5.58) 



(5.59) 



(5.60) 
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at x = Xh, where 



rL 



Yi{y) cos(j7ty/L) dy, 



(5.61) 



etc. Note, however, that the factor in front of the integrals in Eqs. (15.571) and 
(15.611) takes the special value 1 /L for the j = harmonic. 

As before, we truncate the Fourier expansion in the y -direction, and discretize 
in the x-direction, to obtain the set of tridiagonal matrix equations specified in 
Eqs. (15.511) . (15.521) . and (15.531) . We can solve these equations to obtain the Uy, 
and then reconstruct the u(xi,Uj) from Eq. (15.551) . Hence, we have solved the 
problem. 



5.9 The Fast Fourier Transform 



The method outlined in Sect. 15. 71 for solving Poisson's equation in 2-d with simple 
Dirichlet boundary conditions in the y -direction requires us to perform very many 
Fourier-sine transforms: 

2 1-1 

Ff = T ^f k sin(jk7T/n (5.62) 

' k=1 

for j = 0, J, and inverse Fourier-sine transforms: 

J-i 

f, = ^ F kSin(jk7t/J). (5.63) 

k=1 



Here, fj is the value of f(*y) at Uj = j L/J. Thus, Eq. (15.62D is analogous to 
Eqs. (15.461) and (15.491) , whereas Eq. (15.631) can be used to reconstruct the u(xt, Uj) 



from the Uy. Likewise, the method outlined in Sect. 15.81 for solving Poisson's 
equation in 2-d with simple Neumann boundary conditions in the y -direction 
requires us to perform very many Fourier-cosine transforms: 

rf = j + y Y- fk cos ^ kn ^ + ^-T^ 1 (5 - 64) 

' ' k=1 ' 
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for j = 0, J, and inverse Fourier-cosine transforms: 

J 

f, =X F £ cos(jk7r/J). (5.65) 

k=0 

Unfortunately, performing such transforms directly requires 0(J 2 ) arithmetic op- 
erations, which means that they are extremely expensive in terms of cpu resources. 
There is, however, an ingenious algorithm for performing Fourier transforms 
which only takes 0(J In J) arithmetic operations [which is much less than 0(J 2 ) 
operations when J is large]. This algorithm is known as the fast Fourier transform 
or FFT. 



35 



The details of the FFT algorithm lie beyond the scope of this course. Roughly 
speaking, the algorithm works by building up the transform in stages using 2, 4, 
8, 16, etc. grid-points. In this course, we shall employ the best-known publicly 
available FFT library, called the f ftw library!^ to perform all of our Fourier-sine 
and -cosine transforms. Unfortunately, the f ftw library does not directly calcu- 
late Fourier-sine and -cosine transforms H Instead, it calculates complex Fourier 
transforms: 

1 21-1 

Fj = y Y- fk eX P("^ k7T / J ) (5 - 66) 

' k=0 

for j = 0, 2J — 1, and complex inverse Fourier transforms: 

2J-1 

f^^FkexptijkTt/J). (5.67) 

k=0 

Note that fj and Fj are periodic in j with period 2 J. Note, further, that the data- 
sets associated with complex Fourier transforms contain twice as many elements 
as the data-sets associated with sine and cosine transforms. However, we can 
easily convert a sine or cosine transform into a complex transform by extending 
its data-set. Thus, for a sine transform we write: 

f 2 j-j = -fj, (5.68) 

F 2J ) = -Fj, (5.69) 



35 See Numerical recipes in C: the art of scientific computing, W.H. Press, S.A. Teukolsky, W.T. Vettering, and B.R. Flan- 
nery (Cambridge University Press, Cambridge, England, 1992), Sect. 12.2. 
36 See http : //www . f f tw . org 

37 This is the case for version 2 of the library (which is the version used in this course), but not version 3. 
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for ) = 1 , J — 1 , in which case 




2iR 



(5.70) 



Likewise, for a cosine transform we write: 




= fi 



= h 



(5.71) 
(5.72) 



for j = 1 , J — 1 , in which case 



F 



c 



2h. 



(5.73) 



Listed below are a set of wrapper routines which employ the f f tw library to 
perform Fourier-sine and -cosine transforms. 

// FFT.cpp 

// Set of functions to calculate Fourier-cosine and -sine transforms 

// of real data using fftw Fast-Fourier-Transform library. 

// Input/ouput arrays are assumed to be of extent J+l. 

// Uses version 2 of fftw library (incompatible with vs 3). 

#include <fftw.h> 
#include <blitz/array.h> 

using namespace blitz; 

// Calculates Fourier-cosine transform of array f in array F 
void f f t_f orward_cos (Array<double , 1> f, Array<double , 1>& F) 
{ 

// Find J. Declare local arrays, 
int J = f. extent (0) - 1; 
int N = 2 * J; 
fftw_complex f f [N] , FF[N] ; 

// Load and extend data 



c_re (ff[0]) = f(0); c_im (f f [0] ) = 0.; 
c_re (ff[J]) = f(J); c_im (f f [J] ) = 0.; 
for (int j = 1; j < J; 



{ 




c_re (ff[j]) = f(j); c_im (ff[j]) = 0.; 



204 



5. 9 The Fast Fourier Transform 



5 POISSON'S EQUATION 



// Call fftw routine 

fftw_plan p = fftw_create_plan (N, FFTW_FORWARD , FFTW_ESTIMATE) ; 
fftw_one (p, ff, FF) ; 
f f tw_destroy_plan (p) ; 

// Unload data 

F(0) = c_re (FF[0]); F(J) = c_re (FF[J]); 
for (int j = 1; j < J; 
{ 

F(j) = 2. * c_re (FF [j] ) ; 

} 

// Normalize data 

F /= 2. * double (J) ; 

> 

// Calculates inverse Fourier-cosine transform of array F in array f 

void f f t_backward_cos (Array<double , 1> F, Array<double , 1>& f) 

{ 

// Find J. Declare local arrays, 
int J = f. extent (0) - 1; 
int N = 2 * J; 
fftw_complex f f [N] , FF [N] ; 

// Load and extend data 
c_re (FF[0]) = F(0) ; c_im (FF[0]) =0.; 
c_re (FF[J]) = F(J) ; c_im (FF[J]) =0.; 
for (int j = 1; j < J; j++) 
{ 

c_re (FF [j] ) = F(j) / 2.; c_im (FF [j] ) = 0.; 
FF[2*J-j] = FF [j] ; 

} 

// Call fftw routine 

fftw_plan p = fftw_create_plan (N, FFTW_BACKWARD , FFTW_ESTIMATE) ; 
fftw_one (p, FF, ff ) ; 
f f tw_destroy_plan (p) ; 

// Unload data 

f(0) = c_re (ff[0]); f(J) = c_re (f f [J] ) ; 
for (int j = 1; j < J; j++) 
{ 

f(j) = c_re (ff[j]); 

} 
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// Calculates Fourier-sine transform of array f in array F 
void f f t_f orward_sin (Array<double , 1> f, Array<double , 1>& F) 
{ 

// Find J. Declare local arrays, 
int J = f. extent (0) - 1; 
int N = 2 * J; 
fftw_complex f f [N] , FF[N] ; 

// Load and extend data 
c_re (ff[0]) =0.; c_im (f f [0] ) =0.; 
c_re (ff[J]) =0.; c_im (f f [ J] ) =0.; 
for (int j = 1; j < J; j++) 
{ 

c_re (ff[j]) = f(j); c_im (ff[j]) = 0.; 

c_re (ff[2*J-j]) = - f(j); c_im (ff[2*J-j]) = 0.; 

} 

// Call fftw routine 

fftw_plan p = fftw_create_plan (N, FFTW_F0RWARD , FFTW_ESTIMATE) ; 
fftw_one (p, ff, FF) ; 
f f tw_destroy_plan (p) ; 

// Unload data 
F(0) = 0. ; F(J) = 0. ; 
for (int j = 1; j < J; j++) 
{ 

F(j) = - 2. * c_im (FF[j]); 

} 

// Normalize data 

F /= 2. * double (J) ; 

} 

// Calculates inverse Fourier-sine transform of array F in array f 

void f f t_backward_sin (Array<double , 1> F, Array<double , 1>& f) 

{ 

// Find J. Declare local arrays, 
int J = f. extent (0) - 1; 
int N = 2 * J; 
fftw_complex f f [N] , FF [N] ; 

// Load and extend data 

c_re (FF[0]) =0.; c_im (FF[0]) =0.; 

c_re (FF[J]) =0.; c_im (FF[J]) =0.; 
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for (int j = 1; j < J; j++) 
{ 

c_re (FF [j] ) = 0.; c_im (FF [j] ) = - F(j) / 2.; 

c_re (FF[2*J-j]) = 0.; c_im (FF[2*J-j]) = F(j) / 2.; 

} 

// Call fftw routine 

fftw_plan p = fftw_create_plan (N, FFTW_BACKWARD , FFTW_ESTIMATE) ; 
fftw_one (p, FF, ff ) ; 
f f tw_destroy_plan (p) ; 

// Unload data 
f (0) = 0. ; f (J) = 0. ; 
for (int j = 1; j < J; j++) 
{ 

f(j) = c_re (ff[j]); 

} 



5.10 An Example 2-D Poisson Solving Routine 

Listed below is an example 2-d Poisson solving routine which employs the previ- 
ously listed tridiagonal matrix inversion and FFT wrapper routines, as well as the 
Blitz++ library. 

// Poisson2d. cpp 

// Function to solve Poisson' s equation in 2-d: 

// d~2 u / dx~2 + d~2 u / dy~2 = v for xl <= x <= xh and <= y <= L 

// alphaL u + betaL du/dx = gammaL(y) at x=xl 

// alphaH u + betaH du/dx = gammaH(y) at x=xh 

//In y-direction, either simple Dirichlet boundary conditions: 

// u(x,0) = u(x,L) = 

// or simple Neumann boundary conditions: 
// du/dy(x,0) = du/dy(x,L) = 
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// Matrices u and v assumed to be of extent N+2, J+l . 
// Arrays gammaL, gammaH assumed to be of extent J+l. 

// Now, (i,j)th elements of matrices correspond to 

// x_i = xl + i * dx i=0,N+l 

// y_j = j * L / J j=0,J 

// Here, dx = (xh - xl) / (N+l) is grid spacing in x-direction. 
// Now, kappa = pi * dx / L 

// Finally, Neumann=0/1 selects Dirichlet/Neumann bcs in y-direction. 
#include <blitz/array .h> 
using namespace blitz; 

void f f t_f orward_cos (Array<double , 1> f, Array<double , 1>& F) ; 
void f f t_backward_cos (Array<double , 1> F, Array<double , 1>& f ) ; 
void f f t_f orward_sin (Array<double , 1> f, Array<double , 1>& F) ; 
void f f t_backward_sin (Array<double , 1> F, Array<double , 1>& f ) ; 
void Tridiagonal (Array<double , 1> a, Array<double , 1> b, Array<double , 1> c, 
Array<double , 1> w, Array<double , 1>& u) ; 

void Poisson2D (Array<double , 2>& u, Array<double,2> v, 

double alphaL, double betaL, Array<double , 1> gammaL, 
double alphaH, double betaH, Array<double , 1> gammaH, 
double dx, double kappa, int Neumann) 

{ 

// Find N and J. Declare local arrays, 
int N = u. extent (0) - 2; 
int J = u. extent (1) - 1; 

Array<double , 2> V(N+2, J+l), U(N+2, J+l); 
Array<double , 1> GammaL ( J+l ) , GammaH ( J+l ) ; 

// Fourier transform boundary conditions 
if (Neumann) 
{ 

f f t_f orward_cos (gammaL, GammaL) ; 
f f t_f orward_cos (gammaH, GammaH) ; 

} 

else 
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f f t_f orward_sin (gammaL, GammaL) ; 
f f t_f orward_sin (gammaH, GammaH) ; 

} 

// Fourier transform source term 
for (int i = 1; i <= N; i++) 
{ 

Array<double,l> In(J+l) , Out(J+l); 

for (int j = 0; j <= J; In(j) = v(i, j); 

if (Neumann) 

f f t_f orward_cos (In, Out); 
else 

f f t_f orward_sin (In, Out); 
for (int j =0; j <= J ; j++) V(i, j) = Out(j); 

} 

// Solve tridiagonal matrix equations 
if (Neumann) 
{ 

for (int j = 0; j <= J; j++) 
{ 

Array<double,l> a(N+2) , b(N+2), c(N+2), w(N+2) , uu(N+2) ; 

// Initialize tridiagonal matrix 

for (int i = 2; i <= N; i++) a(i) = 1.; 

for (int i = 1; i <= N; i++) 

b(i) = -2. - double (j * j) * kappa * kappa; 
b(l) -= betaL / (alphaL * dx - betaL) ; 
b(N) += betaH / (alphaH * dx + betaH) ; 
for (int i = 1; i <= N-l; i++) c(i) = 1.; 

// Initialize right-hand side vector 
for (int i = 1; i <= N; i++) 

w(i) = V(i, j) * dx * dx; 
w(l) -= GammaL(j) * dx / (alphaL * dx - betaL); 
w(N) -= GammaH(j) * dx / (alphaH * dx + betaH); 

// Invert tridiagonal matrix equation 

Tridiagonal (a, b, c, w, uu) ; 

for (int i = 1; i <= N; i++) U(i, j) = uu(i) ; 

> 
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} 

else 
{ 

for (int j = 1; j < J; j++) 
{ 

Array<double,l> a(N+2) , b(N+2) , c(N+2), w(N+2) , uu(N+2) ; 

// Initialize tridiagonal matrix 

for (int i = 2; i <= N; i++) a(i) = 1.; 

for (int i = 1; i <= N; i++) 

b(i) = -2. - double (j * j) * kappa * kappa; 
b(l) -= betaL / (alphaL * dx - betaL) ; 
b(N) += betaH / (alphaH * dx + betaH) ; 
for (int i = 1; i <= N-l; i++) c(i) = 1.; 

// Initialize right-hand side vector 
for (int i = 1; i <= N; i++) 

w(i) = V(i, j) * dx * dx; 
w(l) -= GammaL(j) * dx / (alphaL * dx - betaL); 
w(N) -= GammaH(j) * dx / (alphaH * dx + betaH); 

// Invert tridiagonal matrix equation 

Tridiagonal (a, b, c, w, uu) ; 

for (int i = 1; i <= N; i++) U(i, j) = uu(i) ; 

> 

for (int i = 1; i <= N ; i++) 
{ 

U(i, 0) = 0. ; U(i, J) = 0. ; 

> 



// Reconstruct solution via inverse Fourier transform 
for (int i = 1; i <= N; i++) 
{ 

Array<double, 1> In(J+l) , 0ut(J+l); 

for (int j = 0; j <= J; j++) In(j) = U(i, j); 

if (Neumann) 

f f t_backward_cos (In, Out); 
else 

f f t_backward_sin (In, Out); 
for (int j = 0; j <= J; j++) u(i, j) = 0ut(j); 
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// Calculate i=0 and i=N+l values 
for (int j =0; j <= J; j++) 
{ 

u(0, j) = (gammaL(j) * dx - betaL * u(l, j)) / 

(alphaL * dx - betaL) ; 
u(N+l, j) = (gammaH(j) * dx + betaH * u(N, j)) / 

(alphaH * dx + betaH) ; 

} 

> 



5.11 An Example Solution of Poisson's Equation in 2-D 

Let us now use the techniques discussed above to solve Poisson's equation in two 
dimensions. Suppose that the source term is 

v(x,-y) = 6x-y(l--y)-2x 3 (5.74) 

for < x < 1 and < y < 1 . The boundary conditions at x = are oc\ = 1 , 
(3l = 0, and j\ = [see Eq. (15. 401) 1, whereas the boundary conditions at x = 1 



are an = 1, |3 h = 0, and yh = y (1 — y) [see Eq. (15. 411) 1. The simple Dirichlet 
boundary conditions u(x, 0) = u(x, 1 ) = are applied at y = and y = 1 . Of 
course, this problem can be solved analytically to give 

u(x,u) =y (1 -y)x\ (5.75) 



Figures [63] and [64] show comparisons between the analytic and finite difference 



solutions for N = J = 64. It can be seen that the finite difference solution mirrors 
the analytic solution almost exactly. 

As a second example, suppose that the source term is 

v(x,u) = -2 (2y 3 - 3y 2 + 1 ) + 6 (1 - x 2 ) (2y - 1 ) (5.76) 

for < x < 1 and < y < 1 . The boundary conditions at x = are <X\ — 1 , |3i, = 0, 
and Yi, = 2y 3 — 3 tj 2 + 1 [see Eq. (15. 401) 1, whereas the boundary conditions atx = 1 
are oc h = 1, (3h = 0, and Yh = [see Eq. (15. 411) 1. The simple Neumann boundary 
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Figure 63: Solution ofPoisson's equation in two dimensions with simple Dirichlet boundary condi- 
tions in the y-direction. The solution is plotted versus xaty — 0.5. The dotted curve (obscured) shows 
the analytic solution, whereas the open triangles show the finite difference solution for N = J = 64. 



0.03 



0.02 



0.01 - 



1 1 1 1 1 1 1 1 jMfiVbi 

A 


IgW | 1 1 1 1 1 1 1 

A 


A 






A 




4 


A 


A 


A 


A 






A 


A 

\ 


- A 


A - 


A 
' A 


A 

A ' 


A 
-A 


A 

A' 


i 

A 


\ 

A 

\ 

. i , . , i . , , 



0.2 



0.4 0.6 

y 



0.8 1 



Figure 64: Solution of Poisson's equation in two dimensions with simple Dirichlet boundary condi- 
tions in the y-direction. The solution is plotted versus y atx = 0.5. The dotted curve (obscured) shows 
the analytic solution, whereas the open triangles show the finite difference solution for N = J = 64. 
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Figure 65: Solution ofPoisson's equation in two dimensions with simple Neumann boundary condi- 
tions in the y-direction. The solution is plotted versus xaty — 0.5. The dotted curve (obscured) shows 
the analytic solution, whereas the open triangles show the finite difference solution for N = J = 64. 



conditions 3u(x,0)/3u = 3u(x,1)/3u = are applied at y = and y — 1. Of 
course, this problem can be solved analytically to give 

u(x,y) = (1 -x 2 ) (2u 3 -3y 2 + 1). (5.77) 

Figures [65] and [66] show comparisons between the analytic and finite difference 
solutions for N = J = 64. It can be seen that the finite difference solution mirrors 
the analytic solution almost exactly. 



5.12 Example 2-D Electrostatic Calculation 



Let us perform an example 2-d electrostatic calculation. Consider a charged wire 
running parallel to the axis of a uniform, hollow, rectangular, conducting channel. 
Suppose that the vertices of the channel lie at (x,u) = (0,0), (0,1), (1,0), and 
(1,1). Suppose, further, that the wire carries a uniform charge per unit length of 
magnitude unity. The electric potential c()(x,u) inside the channel satisfies [see 
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Figure 66: Solution ofPoisson's equation in two dimensions with simple Neumann boundary condi- 
tions in the y-direction. The solution is plotted versus y atx = 0.5. The dotted curve (obscured) shows 
the analytic solution, whereas the open triangles show the finite difference solution for N = J = 64. 



Eq. (E3D] 



— ^2 — + — 9^2 — = v ( x >yJ = -Mx-xoJ ~"yoJ, (5.78) 

where (xo,uo) are the coordinates of the wire. Here, we have conveniently nor- 
malized our units such that the factor eo is absorbed into the normalization. As- 
suming that the box is grounded, the potential is subject to the Dirichlet boundary 
conditions 4> = at x = 0, x = 1 , u = 0, and y = 1 . We require the solution in 
the region < x < 1 and < y < 1 . 



Note that when discretizing Eq. (15.781) the right-hand side becomes 



v(x i ,-y j ) = --^— (5.79) 
ox by 

on the grid-point closest to the wire, with v(xi,Uj) = on the remaining grid- 
points. Here, 6x and by are the grid spacings in the x- and y- directions, respec- 
tively. 

Figures [67] and [68] show the electric potential c|)(x,u) and electric field E = 
— Vc() generated by a wire placed at the center of the channel: i.e., (xo,Uo) = 
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Figure 67: Contour plot of the electric potential generated by a charged wire placed at the center of 
a grounded rectangular channel The wire is located at [x,y) = (0.5, 0.5), whereas the channel walls 
are at x = 0, x = 1, y = 0, and y = 1. Calculation performed with N = J = 128. 
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Figure 68: Vector plot showing the direction of the electric field generated by a charged wire placed 
at the center of a grounded rectangular channel. The wire is located at [x,y] = (0.5, 0.5), whereas 
the channel walls are at x = 0, x = 1, y = 0, and y = 1. Calculation performed with N = J = 128. 
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Figure 69: Contour plot of the electric potential generated by a charged wire offset from the center 
of a grounded rectangular channel. The wire is located at [x } y] = (0.25,0.5), whereas the channel 
walls are at x = 0, x = 1, y = 0, and y = 1. Calculation performed with N = J = 128. 

[0.5,0.5). The calculation was performed with the previously listed 2-d Poisson 
solver using N = J = 128. 

Figures [69] and [70] show the electric potential cj)(x,y) and electric field E = 
— V4> generated by a wire offset from the center of the channel: i.e., (xo,yo) = 
(0.25,0.5). The calculation was performed with the previously listed 2-d Poisson 
solver using N = J = 128. 



5.13 3-D Problems 



The techniques discussed in Sects. 15.71 and 15.81 for solving Poisson's equation in 



two dimensions with a restricted class of boundary conditions can easily be gen- 
eralized to three dimensions. In the 3-d case, it is necessary to Fourier transform 
in two directions (the y and z directions, say) in order to reduce the problem 
to a system of uncoupled tridiagonal matrix equations. These equations can be 
inverted in the usual manner, and the solution can then be reconstructed via a 
double inverse Fourier transform. 
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Figure 70: Vector plot showing the direction of the electric field generated by a charged wire offset 
from the center of a grounded rectangular channel. The wire is located at (x,y ) = (0.25, 0.5), whereas 
the channel walls are at x = 0, x = 1, y = 0, and y = 1. Calculation performed with N = J = 128. 
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6 The Diffusion Equation 



6.1 Introduction 



The diffusion equation 

8T,,|t) =D^r,t), (6.1) 



qdS, (6.3) 



at 

where D > is the (uniform) coefficient of diffusion, describes many interesting 
physical phenomena. For instance, in heat conduction we can write 

q = - K VT, (6.2) 

where q is the heat flux, T the temperature, and k the coefficient of thermal con- 
ductivity. The above equation merely states that heat flows down a temperature 
gradient. In the absence of sinks or sources of heat, energy conservation requires 
that 

_9Q 

at 

where Q is the thermal energy contained in some volume V bounded by a closed 
surface S. The above equation states that the rate of decrease of the thermal 
energy content of volume V equals the instantaneous heat flux flowing across its 
boundary. However, Q = J c T dV, where c is the heat capacity per unit volume. 
Making use of the previous equations, as well as the divergence theorem, we 
obtain the following diffusion equation for the temperature: 

^ = D V 2 T, (6.4) 
ot 

where D = k/c In a typical heat conduction problem, we are given the tem- 
perature T(r,to) at some initial time to, and then asked to evaluate T(r, t) at all 
subsequent times. Such a problem is known as an initial value problem. The spa- 
tial boundary conditions can be either of type Dirichlet (i.e., T specified on the 
boundary), type Neumann (i.e., VT specified on the boundary), or some combi- 
nation. 
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6.2 1-D Problem with Mixed Boundary Conditions 



Consider the solution of the diffusion equation in one dimension. Suppose that 

3T(x,t) 3 2 T(x,t) 

~^T~ = C6 - 5) 

for x\ < x < xh, subject to the mixed spatial boundary conditions 

cx l (t)T(x,t) + (3 L (t) = yi (t), (6.6) 

ox 

at x = X\, and 

a K (t) T(x, t) + (3 h (t) = y h (t) , (6.7) 

ox 

at x = Xh- Here, <X\, $\, etc., are known functions of time. Of course, T(x, to) must 
be specified at some initial time to- 

Equation (16.51) needs to be discretized in both time and space. In time, we 
discretize on the equally spaced grid 

t n = t + u6t, (6.8) 

where 6t is the time-step. Adopting a simple first-order differencing scheme, 
Eq. (16.51) becomes 

T(x,t n+ ,}-T(*,t n) =D 3ZT(xt n ) 
ot ox 1 

In space, we discretize on the usual equally spaced grid-points specified in Eq. (15.71) , 
and approximate d 2 /dx 2 via the second-order, central difference scheme intro- 
duced in Eq. (15. 8D . The spatial boundary conditions are discretized in a similar 
manner to Eqs. (ET27D and (15^81) . Thus, Eq. (|6T9l) yields 

1 k ~ D 1 ~X 1+1 , (6.10) 



6t (6x) 

or 

-n+l TTL 1 /"« /-ru -) TTl 1 ttl 



T- + ' ^ + ^-21? + ^) (6.11) 
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for i = 1 , IN, where = T(xj., t n ), and C = D 6t/(6x) 2 . The discretized boundary 
conditions take the form 

T ° = «P6X- P f ' (6 - 12) 



T N +i = — — r^^» C6.13J 



where yj 1 = yi(t n ), etc. The discretization scheme outlined above is termed first- 
order in time and second-order in space. 



Equations (16.11D - (16.13D constitute a fairly straightforward iterative scheme 



which can be used to evolve the T(x, t) in time. 
6.3 An Example 1-D Diffusion Equation Solver 

Listed below is an example 1-d diffusion equation solving routine which makes 
use of the Blitz++ library. 

// Dif fusionlD . cpp 

// Function to evolve diffusion equation in 1-d: 

// dT / dt = D d"2 T / dx"2 for xl <= x <= xh 

// alpha_l T + beta_l dT/dx = gamma_l at x=xl 

// alpha_h T + beta_h dT/dx = gamma_h at x=xh 

// Array T assumed to be of extent N+2. 

// Now, ith element of array corresponds to 

// x_i = xl + i * dx i=0,N+l 

// Here, dx = (xh - xl) / (N+l) is grid spacing. 

// Function evolves T by single time-step. 

// C = D dt / dx~2, where dt is time-step. 
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// Uses explicit scheme. 

#include <blitz/array .h> 

using namespace blitz; 

void DiffusionlD (Array<double , 1>& T, 

double alpha_l, double beta_l, double gamma_l, 
double alpha_h, double beta_h, double gamma_h, 
double dx, double C) 

{ 

// Set N. Declare local array, 
int N = T. extent (0) - 2; 
Array<double , 1> TO (N+2) ; 

// Evolve T 
TO = T; 

for (int i = 1; i <= N; i++) 

T(i) += C * (TO(i-l) - 2. * T0(i) + T0(i+1)); 

// Set boundary conditions 

T(0) = (gamma_l * dx - beta_l * T(l)) / (alpha_l * dx - beta_l) ; 
T(N+1) = (gamma_h * dx + beta_h * T(N) ) / (alpha_h * dx + beta_h) ; 



6.4 An Example 1-D Solution of the Diffusion Equation 



Let us now solve the diffusion equation in 1-d using the finite difference technique 
discussed above. We seek the solution of Eq. (16.51) in the region — xo < x < xo, 
subject to the initial condition 



T(x,t ) = exp! 



—x 



4Dt ; ' 

where to > 0. The spatial boundary conditions are 



(6.14) 



T(±x ,t) 



t 

NT expi 



4Dt 



(6.15) 
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Figure 71: Diffusive evolution of a 1-d Gaussian pulse. Numerical calculation performed using D = 1, 
xo = 5; St = 4 x 10~ 3 , and N = 100. The pulse is evolved from t = 0.1 to t = 1. The solid curve 
shows the initial condition at t = 0.1, the dashed curve the numerical solution at t = 1, and the 
dotted curve (obscured by the dashed curve) the analytic solution att = 1. 

Of course, we can solve this problem analytically to give 



T(x,t) 



to / -x 2 



w exi w- < 6 - 16 > 



Note that the above equation describes a Gaussian pulse which gradually de- 
creases in height and broadens in width in such a manner that its area is con- 
served. The width of the pulse varies approximately as 

Ax-VdI. (6.17) 

Moreover, the pulse approaches a 6 -function as t — > 0. 



Figure [TJ] shows a comparison between the analytic and numerical solutions 
for a calculation performed using D = 1, xo = 5, to = 0.1, 6t = 4 x 10 3 , and 
N = 100. It can be seen that the analytic and numerical solutions are in excellent 
agreement. 

It is reasonable to expect that as N increases at fixed 6t (i.e., the spatial resolu- 
tion increases at fixed temporal resolution) the numerical solution should become 
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Figure 72: Diffusive evolution of a 1-d Gaussian pulse. Numerical calculation performed using D = 1, 
x = 5, 6t = 4 x 10~ 3 , and N = 125. The simulation is started at t = 0.1. The top-left, top-right, 
bottom-left, and bottom-right panels show the solution att = 0.45, t = 0.46, t = 0.47, and t = 0.48, 
respectively. 
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more and more accurate. This is indeed the case — at least, until N exceeds a crit- 
ical value. Beyond this value, there is a catastrophic breakdown in the numerical 
solution. This breakdown is illustrated in Fig. [721 It can be seen that the solu- 
tion develops rapidly growing short-wavelength oscillations. Indeed, the solution 
eventually becomes effectively infinite. Let us investigate this unusual and rather 
disturbing phenomenon. 



6.5 von Neumann Stability Analysis 

Clearly, our simple finite difference algorithm for solving the 1-d diffusion equa- 
tion is subject to a numerical instability under certain circumstances. Let us try 
to establish when this instability occurs. Consider the time evolution of a single 
Fourier mode of wave-number k: 

T(x,t) =t(t)e ikx . (6.18) 



Substitution of the above expression into our finite difference scheme (16.111) 
yields 

fu+1 e ik Xn = fn e ikx n j-| +C ( e -ik6x_ 2 + e +ik5x' ) ] ^ (6J9) 

f n+1 = A t n , (6.20) 



or 



where 

A = 1 — 2 C ( 1 — cos k 6x) = 1 — 4 C sin 2 (k6x/2). (6.21) 

Thus, the amplitude of the Fourier mode is amplified by a factor A at each time- 
step. In order for the differencing scheme to be stable, the modulus of this ampli- 
fication factor must be less than unity for all possible values of k. Now, the largest 
possible value of sin (k6x/2) is unity: hence, the wave-length corresponding to 
this value is that of the most unstable Fourier mode. In fact, the most unstable 
mode possesses a wave-length which is half the grid-spacing: i.e., A = bx/2. It 
follows from Eq. (16.21D that this mode is stable provided 

C < 1 (6.22) 
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Finally, from the definition of C, our stability condition can be written 

« < (6.23) 

Note that C = 0.408 for the stable calculation shown in Fig. [7H whereas C = 
0.635 for the unstable calculation shown in Fig. [72]. Incidentally, the type of 
stability analysis outlined above is called von Neumann stability analysis. Note 
that the neglect of the spatial boundary conditions in the above calculation is 
justified because the unstable modes vary on very small length-scales which are 
typically of order the grid spacing. 

According to Eq. (16.23D . our finite difference scheme for solving the 1-d diffu- 
sion equation is only stable provided that the time-step remains below some crit- 
ical value. Note that this critical value scales like the square of the grid-spacing. 
This is a very unfavorable scaling, since it implies that a doubling of the spatial 
resolution requires a simultaneous reduction in the time-step by a factor of four 
in order to maintain numerical stability. Certainly, the above constraint limits us 
to absurdly small time-steps in high resolution calculations. Is there any way of 
overcoming this constraint? 



6.6 The Crank-Nicholson Scheme 

Let us revisit our temporal differencing scheme: 

T(x,t n+ ,)-T(x,t n ) = D 3'T(x y + 
ot ox z 

Note that the right-hand side is evaluated entirely at the start of the time-step: 
i.e., at t n . This type of temporal differencing scheme is termed an explicit scheme. 
Now, explicit schemes are very straightforward to implement, but are also notori- 
ously prone to numerical instabilities. Fortunately, we can often overcome these 
instabilities by making our differencing scheme implicit in nature. An implicit 
scheme is one in which the right-hand side is evaluated partly (or wholly) at the 
end of the time-step: i.e., at t n+ i. Unfortunately, implicit schemes are generally a 
great deal more complicated to implement than explicit schemes. 
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The well-known Crank-Nicholson implicit method for solving the diffusion 
equation involves taking the average of the right-hand side between the begin- 
ning and end of the time-step. In other words, 

T(x,t n+ i)-T(x,t a ) D 3 2 T(x,t n ) D 3 2 T(x,t n+1 ) 2 

6t = 2 ^x^ + 2 ^ + ° m ■ (6 - 25) 

As indicated by the error term, this method is actually second-order in time. 
Adopting our usual spatial differencing scheme, the above expression yields 

T7 +1 - § (T£V - 2T^ +1 + T^V) = T? + \ (TT-i - 21? + 1? +1 ) . (6.26) 

When we perform a von Neumann stability analysis of the above scheme, we 
obtain the following expression for the amplification factor: 

A = 1 - 2Csfa ; (kS * /2) . (6.27) 
1 +2C sin 2 (k6x/2) 

Note that |A| < 1 for all values of k. It follows that the Crank-Nicholson scheme is 
unconditionally stable. Unfortunately, Eq. (I6.26D constitutes a tridiagonal matrix 
equation linking the T^ 1 and the T?\ Thus, the price we pay for the high accuracy 
and unconditional stability of the Crank-Nicholson scheme is having to invert a 
tridiagonal matrix equation at each time-step. Usually, this price is well worth 
paying. 



6.7 An Improved 1-D Diffusion Equation Solver 

Listed below is an improved 1-d diffusion equation solver which uses the Crank- 
Nicholson scheme, as well as the previous listed tridiagonal matrix solver and the 
Blitz++ library. Note the great structural si milar ity between this solver and the 
previously listed 1-d Poisson solver (see Sect. [53]). 

// CrankNicholsonlD . cpp 

// Function to evolve diffusion equation in 1-d: 
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// dT / dt = D d~2 T / dx~2 for xl <= x <= xh 

// alpha_l T + beta_l dT/dx = gamma_l at x=xl 

// alpha_h T + beta_h dT/dx = gamma_h at x=xh 

// Array T assumed to be of extent N+2. 

// Now, ith element of array corresponds to 

// x_i = xl + i * dx i=0,N+l 

// Here, dx = (xh - xl) / (N+l) is grid spacing. 

// Function evolves T by single time-step. 

// C = D dt / dx~2, where dt is time-step. 

// Uses Crank-Nicholson implicit scheme. 



#include <blitz/array.h> 
using namespace blitz; 

void Tridiagonal (Array<double , 1> a, Array<double , 1> b, Array<double , 1> c, 

Array<double , 1> w, Array<double , 1>& u) ; 

void CrankNicholsonlD (Array<double , 1>& T, 

double alpha_l, double beta_l, double gamma_l, 
double alpha_h, double beta_h, double gamma_h, 
double dx, double C) 

{ 

// Find N. Declare local arrays, 
int N = T. extent (0) - 2; 

Array<double,l> a(N+2) , b(N+2) , c(N+2), w(N+2) ; 



// Initialize tridiagonal matrix 

for (int i = 2; i <= N; i++) a(i) = - 0.5 * C; 

for (int i = 1; i <= N; i++) b(i) = 1. + C; 

b(l) += 0.5 * C * beta_l / (alpha_l * dx - beta_l) ; 

b(N) -= 0.5 * C * beta_h / (alpha_h * dx + beta_h) ; 

for (int i = 1; i <= N-l; i++) c(i) = - 0.5 * C; 



// Initialize right-hand side vector 
for (int i = 1; i <= N; i++) 
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w(i) = T(i) + 0.5 * C * (T(i-l) - 2. * T(i) + T(i+1)); 
w(l) += 0.5 * C * gamma_l * dx / (alpha_l * dx - beta_l) ; 
w(N) += 0.5 * C * gamma_h * dx / (alpha_h * dx + beta_h) ; 

// Invert tridiagonal matrix equation 
Tridiagonal (a, b, c, w, T) ; 

// Calculate i=0 and i=N+l values 

T(0) = (gamma_l * dx - beta_l * T(l)) / 

(alpha_l * dx - beta_l) ; 
T(N+1) = (gamma_h * dx + beta_h * T(N) ) / 

(alpha_h * dx + beta_h) ; 



6.8 An Improved 1-D Solution of the Diffusion Equation 



Let us now solve the simple diffusion problem introduced in Sect. 16.41 with the 



above listed Crank-Nicholson routine. Figure [73] shows a comparison between 



the analytic and numerical solutions for a calculation performed using D = 1, 
xo = 5, to = 0.1, 6t = 0.1, and N = 100. It can be seen that the analytic and 
numerical solutions are in excellent agreement. Note, however, that the time- 
step used in this calculation (i.e., 6t = 0.1) is much larger than that used in our 
previous calculation (i.e., 6t = 4 x 10 3 ), which employed an explicit differencing 
scheme — see Fig. [71]. According to Eq. (16.231) . an explicit scheme is limited to 
time-steps less than about 5 x 1 3 for the problem under investigation. Thus, we 
have been able to exceed this limit by a factor of 20 with our implicit scheme, yet 
still maintain numerical stability. Note that our Crank-Nicholson scheme is able 
to obtain accurate results with a time-step as large as 0.1 because it is second-order 
in time. 
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Figure 73: Diffusive evolution of a 1-d Gaussian pulse. Numerical calculation performed using D = 1, 
x = 5, 5t = 0.1, and N = 100. The pulse is evolved from t = 0.1 to t = 1. The solid curve shows the 
initial condition at t = 0.1, the dashed curve the numerical solution at t = 1, and the dotted curve 
(obscured by the dashed curve) the analytic solution att= 1 . 



6.9 2-D Problem with Dirichlet Boundary Conditions 



Let us consider the solution of the diffusion equation in two dimensions. Suppose 
that 

9T(x,u,t) 3 2 T(x,-y,t) 3 2 T(x,-y,t) 

^^ = P ^ +p w ' (6 - 28) 

for xi < x < Xh, and < u < L. Suppose that T(x,y, t) satisfies mixed boundary 
conditions in the x-direction: 



at x = xi, and 



ai (t) T(x,y,t) + Ut) dT[x ' V ' t] = Yl (y,t), (6.29) 

ox 



«h(t) T(x,y,t) + p h (t) dT[x > y > t] =y h [y t t) t (6.30) 

ox 



at x = Xk- Here, oc\, (3i, etc., are known functions of t, whereas y L , y h are known 
functions of y and t. Furthermore, suppose that T(x,y,t) satisfies the following 
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simple Dirichlet boundary conditions in the y -direction: 

T(x,0,t) = T(x,L,t) =0. (6.31) 

As before, we discretize in time on the uniform grid t n = to + n 6t, for n = 
0, 1 , 2, ■ ■ •. Furthermore, in the x-direction, we discretize on the uniform grid x\ = 
Xi + i6x, for i = 0, N + 1, where 6x = (xh — *i)/(N + 1 ). Finally, in the y -direction, 
we discretize on the uniform grid ijj = j by, for j = 0, J, where by = L/J. Adopting 
the Crank-Nicholson temporal differencing scheme discussed in Sect. 16.61 and the 
second-order spatial differencing scheme outlined in Sect. 15. 2[ Eq. (16.281) yields 

Tff'-Tj P Taj-2Ty + '+^' D/3 2 T\ n+ '_ 
St 2 (5x)2 2 W)^ 

where Ty = T(xt,Uj, t n ). The discretized boundary conditions take the form 

T n _ yl^x-PTHj (633) 



plus 

l i,0 = 'ij 

Here, < = cx L (t n ), etc., and yft = y x {y h t n ), etc. 



T/o — TTt — 0. (6.35) 



Adopting the Fourier method introduced in Sect. 15.71 we write the Ty in terms 
of their Fourier-sine harmonics: 

J 

T 5 = L T X sin ^ k7r /^ (6 - 36) 

k=0 



which automatically satisfies the boundary conditions (16.351) . The above expres- 
sion can be inverted to give (see Sect. 15. 91) 

2 1 

f 5 = .^L^smljkn/J). (6.37) 

' k=0 
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When Eq. (I6.32D is written in terms of the T^, it reduces to 

- \ 1M1 + { i + c ( i + f k 2 /2) } t3 +1 - j tK! = 

JtC-u + {1 - C (1 + j 2 k 2 /2)} 13 + § t^j, (6.38) 

for i = 1 , IN, and j = 0, J. Here, C = D 6t/(6x) 2 , and k = n 6x/L. Moreover, the 
boundary conditions (I6.33D and (16.341) yield 

r^x-^tfi 



'o,j 



fu 



<6x- |3[ 



H 'N,j 



a£5x + |3£ 



where 



(6.39) 
(6.40) 

(6.41) 



k=0 



etc. Equations (16.381) — (16.40D constitute a set of J + 1 uncoupled tridiagonal 
matrix equations for the t^ +1 , with one equation for each separate value of j. 

In order to advance our solution by one time-step, we first Fourier transform 
the and the boundary conditions, according to Eqs. (I6.37D and (I6.41D . Next, 
we invert the J + 1 tridiagonal equations (16.38D — (16.401) to obtain the Ty +1 . Fi- 
nally, we reconstruct the via Eq. (16.36D . 



6.10 2-D Problem with Neumann Boundary Conditions 



Let us replace the Dirichlet boundary conditions (16.351) by the following simple 
Neumann boundary conditions: 

9T(x,0,t) 3T(x,L,t) 

* = * = °- (6.42) 

The method of solution outlined in the previous section is unaffected, except 
that the Fourier-sine transforms are replaced by Fourier-cosine transforms — see 



Sects. 5.8 and 5.9 
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6.11 An Example 2-D Diffusion Equation Solver 

Listed below is an example 2-d diffusion equation solver which uses the Crank- 
Nicholson scheme, as well as the previous listed tridiagonal matrix solver and the 
Blitz++ library. Note the great structural si milarit y between this solver and the 
previously listed 2-d Poisson solver (see Sect. 15.101) . 

// CrankNicholson2D . cpp 

// Function to evolve diffusion equation in 2-d: 

// dT / dt = D d"2 T / dx~2 + D d"2 T / dy~2 for xl <= x <= xh 
// and <= y <= L 

// alphaL T + betaL dT/dx = gammaL(y) at x=xl 

// alphaH T + betaH dT/dx = gammaH(y) at x=xh 

//In y-direction, either simple Dirichlet boundary conditions: 

// T(x,0) = T(x,L) = 

// or simple Neumann boundary conditions: 

// dT/dy(x,0) = dT/dy(x,L) = 

// Matrix T assumed to be of extent N+2, J+l. 

// Arrays gammaL, gammaH assumed to be of extent J+l. 

// Now, (i,j)th elements of matrices correspond to 

// x_i = xl + i * dx i=0,N+l 



// Here, dx = (xh - xl) / (N+l) is grid spacing in x-direction. 
// Now, C = D dt / dx~2, and kappa = pi * dx / L 

// Finally, Neumann=0/1 selects Dirichlet/Neumann bcs in y-direction. 
// Uses Crank-Nicholson scheme. 
#include <blitz/array .h> 



// y_j = j * L / J 



j=0,J 
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using namespace blitz; 

void f f t_f orward_cos (Array<double , 1> f, Array<double , 1>& F) ; 
void f f t_backward_cos (Array<double , 1> F, Array<double , 1>& f ) ; 
void f f t_f orward_sin (Array<double , 1> f, Array<double , 1>& F) ; 
void f f t_backward_sin (Array<double , 1> F, Array<double , 1>& f ) ; 
void Tridiagonal (Array<double , 1> a, Array<double , 1> b, Array<double , 1> c, 
Array<double , 1> w, Array<double , 1>& u) ; 

void CrankNicholson2D (Array<double , 2>& T, 

double alphaL, double betaL, Array<double , 1> gammaL, 
double alphaH, double betaH, Array<double , 1> gammaH, 
double dx, double C, double kappa, int Neumann) 

{ 

// Find N and J. Declare local arrays, 
int N = T. extent (0) - 2; 
int J = T.extent(l) - 1; 

Array<double , 2> TT(N+2, J+l) , V(N+2, J+l) ; 
Array<double , 1> GammaL ( J+l ) , GammaH ( J+l ) ; 

// Fourier transform T 
for (int i = 0; i <= N+l; i++) 
{ 

Array<double,l> In(J+l) , Out (J+l); 

for (int j =0; j <= J ; j++) In(j) = T(i, j); 

if (Neumann) 

f f t_f orward_cos (In, Out); 
else 

f f t_f orward_sin (In, Out); 
for (int j = 0; j <= J; j++) TT(i, j) = Out(j); 

} 

// Fourier transform boundary conditions 
if (Neumann) 
{ 

f f t_f orward_cos (gammaL, GammaL); 
f f t_f orward_cos (gammaH, GammaH) ; 

} 

else 
{ 

f f t_f orward_sin (gammaL, GammaL); 
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f f t_f orward_sin (gammaH, GammaH) ; 

} 

// Construct source term 
for (int i = 1; i <= N; i++) 
for (int j =0; j <= J; j++) 
VU, j) = 

0.5 * C * TT(i-l, j) + 

(1. - C * (1. +0.5* double (j * j) * kappa * kappa)) * TT(i, j) + 
0.5 * C * TT(i+l, j); 

// Solve tridiagonal matrix equations 
if (Neumann) 
{ 

for (int j = 0; j <= J; j++) 
{ 

Array<double,l> a(N+2) , b(N+2), c(N+2), w(N+2) , u(N+2) ; 

// Initialize tridiagonal matrix 

for (int i = 2; i <= N; i++) a(i) = - 0.5 * C; 

for (int i = 1; i <= N; i++) 

b(i) =1. + C * (1. + 0.5 * double (j * j) * kappa * kappa); 
b(l) += 0.5 * C * betaL / (alphaL * dx - betaL) ; 
b(N) -= 0.5 * C * betaH / (alphaH * dx + betaH) ; 
for (int i = 1; i <= N-l; i++) c(i) = - 0.5 * C; 

// Initialize right-hand side vector 
for (int i = 1; i <= N; i++) 

w(i) = V(i, j); 
w(l) += 0.5 * C * GammaL(j) * dx / (alphaL * dx - betaL); 
w(N) += 0.5 * C * GammaH(j) * dx / (alphaH * dx + betaH); 

// Invert tridiagonal matrix equation 
Tridiagonal (a, b, c, w, u) ; 

for (int i = 1; i <= N; i++) TT(i, j) = u(i) ; 

> 

} 

else 
{ 

for (int j = 1; j < J; j++) 
{ 

Array<double,l> a(N+2) , b(N+2), c(N+2), w(N+2) , u(N+2) ; 

// Initialize tridiagonal matrix 

for (int i = 2; i <= N; i++) a(i) = - 0.5 * C; 



234 



6.11 An Example 2-D Diffusion Equation Solver 



6 THE DIFFUSION EQUATION 



for (int i = 1; i <= N; i++) 

b(i) = 1. + C * (1. + 0.5 * double (j * j) * kappa * kappa); 
b(l) -= betaL / (alphaL * dx - betaL) ; 
b(N) += betaH / (alphaH * dx + betaH) ; 
for (int i = 1; i <= N-l; i++) c(i) = - 0.5 * C; 

// Initialize right-hand side vector 
for (int i = 1; i <= N; i++) 

w(i) = V(i, j) ; 
w(l) += 0.5 * C * GammaL(j) * dx / (alphaL * dx - betaL); 
w(N) += 0.5 * C * GammaH(j) * dx / (alphaH * dx + betaH); 

// Invert tridiagonal matrix equation 
Tridiagonal (a, b, c, w, u) ; 

for (int i = 1; i <= M; i++) TT(i, j) = u(i) ; 

} 

for (int i = 1; i <= N ; i++) 
{ 

TT(i, 0) = 0. ; TT(i , J) = 0. ; 

> 

} 

// Reconstruct solution 
for (int i = 1; i <= N; i++) 
{ 

Array<double,l> In(J+l) , 0ut(J+l); 

for (int j = 0; j <= J; j++) In(j) = TT(i , j); 

if (Neumann) 

f f t_backward_cos (In, Out); 
else 

f f t_backward_sin (In, Out); 
for (int j =0; j <= J ; j++) T(i, j) = 0ut(j); 



// Calculate i=0 and i=N+l values 
for (int j =0; j <= J; j++) 
{ 

T(0, j) = (gammaL(j) * dx - betaL * T(l, j)) / 

(alphaL * dx - betaL) ; 
T(N+1, j) = (gammaH(j) * dx + betaH * T(N, j)) / 

(alphaH * dx + betaH) ; 
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6.12 An Example 2-D Solution of the Diffusion Equation 

Let us now solve the diffusion equation in 2-d using the finite difference technique 
discussed above. We seek the solution of Eq. (16.281) in the region < x < 1 and 



< y < 1, subject to the following initial condition at t = 0: 

T(x,u,0) = 1 for |x-0.5|<0.1 and \y - 0.5| < 0.1, 

T(x,u,0) = otherwise. (6.43) 

The boundary conditions are simply T(0,u) = T(l,u) = T(x,0) = T(x, 1) = 0. 

Figure [741 shows the evolution of T(x,y,t) for a calculation performed with 
the previously listed 2-d diffusion equation solver using D = 1, 6t = 10 4 , and 
M = J = 128. 

6.13 3-D Problems 

The techniques discussed above for solving the diffusion equation in two dimen- 
sions with a restricted class of boundary conditions can easily be generalized to 
three dimensions. In the 3-d case, it is necessary to Fourier transform in two di- 
rections (the y and z directions, say) in order to reduce the problem to a system 
of uncoupled tridiagonal matrix equations. These equations can be inverted in 
the usual manner, and the solution can then be reconstructed via a double inverse 
Fourier transform. 
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Figure 74: Diffusion in two dimensions. Numerical calculation performed using D = 1, 6t = 10 4 , 
and N = J = 128. Density plots o/T(x,y,t) are shown at t = 0.000 (top-left), t = 0.001 (top- 
right), t = 0.002 (middle-left), t = 0.003 (middle-right), t = 0.004 (bottom-left), and t = 0.005 
(bottom-right). 



237 



7 THE WAVE EQUATION 



7 The Wave Equation 
7.1 Introduction 

The wave equation, which in one dimension takes the form 

a ' £ 2 8 ^ ,,,, 
3? = C W (7 - 1} 

occurs so frequently in physics that it is not necessary to enumerate examples. 
Here, £, is usually some sort of displacement or perturbation, whereas c is the 
(constant) wave speed. The wave equation possesses the formal solution 

£,(x,t) = F(x-ct) + G(x + ct), (7.2) 

where F and G are arbitrary functions. The above solution represents arbitrar- 
ily shaped wave pulses propagating with speed c in the +x and — x directions, 
respectively, without changing shape. 

The wave equation, which is second-order in space and time, can be written 
as two coupled first-order equations by defining the new variables v = 3£,/3t and 
= — c 3£,/3x. Expressing Eq. 07. ID in terms of these new variables, we obtain 

Yt +c V, = °> (7 - 3) 

^ + c^ = 0. (7.4) 
3t 3x 

Note that when solving the wave equation numerically it is generally preferable 
to write it as a set of coupled first-order equations, as shown above. 



7.2 The 1-D Advection Equation 

The wave equation is closely related to the so-called advection equation, which in 
one dimension takes the form 

— = -v — (7 5) 

3t 3x' 
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This equation describes the passive advection of some scalar field u(x, t) carried 
along by a flow of constant speed v. Since the advection equation is somewhat 
simpler than the wave equation, we shall discuss it first. The advection equation 
possesses the formal solution 

u(x,t) = F(x-vt), (7.6) 

where F is an arbitrary function. This solution describes an arbitrarily shaped 
pulse which is swept along by the flow, at constant speed v, without changing 
shape. 



We seek the solution of Eq. (17.51) in the region x\ < x < x^, subject to the 



simple Dirichlet boundary conditions u{x\) = u(xh) = 0. As usual, we discretize 
in time on the uniform grid t n = to + u6t, for n = 0, 1 , 2, • ■ - . Furthermore, in 
the x-direction, we discretize on the uniform grid Xi = x\ + i 6x, for i = 0, N + 1 , 
where 6x = (xk— xi) /(N + 1 ). Adopting an explicit temporal differencing scheme, 
and a centered spatial differencing scheme, Eq. (17.51) yields 

1 i n+1 — 1 i n 1 i n — 1 i n 

Ui j- Ui =-v Ui+ ' Ui -' , C7.7) 
ot 2 ox 

where u- 1 = u(xi, t n ). The above equation can be rewritten 

< +1 =<-^ « +1 -u?_i), (7.8) 

where C = v 6t/6x. 

Let us perform a von Neumann stability analysis of the above differencing 
scheme. Writing u(x,t) = H(t) exp(ikx), we obtain u^ +1 = Aft- 1 , where 

A = 1 - iC sin(k6x). (7.9) 

Note that 

|A| 2 = 1 + C 2 sin 2 (k6x) > 1. (7.10) 

Thus, the magnitude of the amplification factor is greater than unity for all k. 
This implies, unfortunately, that the simple differencing scheme (17.81) is uncondi- 
tionally unstable. 
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7.3 The Lax Scheme 



The instability in the differencing scheme (17.81) can be fixed by replacing u- 1 on 



the right-hand side by the spatial average of u taken over the neighbouring grid 
points. Thus, we obtain 

< +] = \ «,+<_,)- \ «,-<_,), (7.11) 

which is known as the Lax scheme. A von Neumann stability analysis of the Lax 
scheme yields the following expression for the amplification factor: 

A = cos(k6x) — i C sin(k6x). (7.12) 

Now 

|A| 2 = 1 _ (1 _ C 2 ) sin 2 (k6x). (7.13) 

It follows that the Lax scheme is unconditionally stable (i.e., |A| < 1 for all k), 
provided that C < 1 . From the definition of C, the inequality C < 1 can also be 
written 

6x 

5t<— . (7.14) 
v 

This is the famous Courant-Friedrichs-Lewy (or CFL) stability criterion. In fact, all 
stable explicit differencing schemes for solving the advection equation are subject 
to the CFL constraint, which determines the maximum allowable time-step. 

Listed below is a routine which solves the 1-d advection equation via the Lax 
method. 

// LaxlD.cpp 

// Function to evolve advection equation in 1-d: 

// du / dt + v du / dx = for xl <= x <= xh 

// u = at x=xl and x=xh 

// Array u assumed to be of extent N+2. 

// Now, ith element of array corresponds to 
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// x_i = xl + i * dx i=0,N+l 

// Here, dx = (xh - xl) / (N+l) is grid spacing. 

// Function evolves u by single time-step. 

// C = v dt / dx, where dt is time-step. 

// Uses Lax scheme. 

#include <blitz/array .h> 

using namespace blitz; 

void LaxlD (Array<double , 1>& u, double C) 
{ 

// Set N. Declare local array, 
int N = u. extent (0) - 2; 
Array<double , 1> uO (N+2) ; 

// Evolve u 
uO = u; 

for (int i = 1; i <= N; i++) 

u(i) = 0.5 * (uO(i+l) + uO(i-l)) - 0.5 * C * (uO(i+l) - uO(i-l)); 

// Set boundary conditions 
u(0) = 0.; 
u(N+l) = 0. ; 

} 

Figure [75] shows an example calculation which uses the above routine to ad- 
vect a Gaussian pulse. The initial condition is 

u(x,0) = exp[-1 00 (x-0.5) 2 ], (7.15) 

and the calculation is performed with v = 1, 6t = 2.49 x 10 3 , and N = 200. 
Furthermore, xl = vt and Xk = 1 + vt. Note that C = 0.5 for these parameters. 
It can be seen that the pulse is advected at the correct speed: i.e., the pulse 
appears approximately stationary when plotted versus x — v t. Unfortunately, the 
pulse does not remain the same shape (as it should). Instead, the pulse becomes 
gradually lower and wider as it propagates, and eventually diffuses away entirely. 

It is clear, from the above calculation, that the Lax scheme introduces a spu- 
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Figure 75: Advection of a 1-d Gaussian pulse. Numerical calculation performed using v = ), 6t = 
2.49 x 1 -3 , and N = 200. The solid curve shows the initial condition at t = 0, the short-dashed 
curve the numerical solution at t = \, the long-dashed curve the numerical solution at t = 2, and the 
dot-dashed curve the numerical solution at t = 3. 

rious dispersion effect into the advection problem. We can understand the ori- 
gin of this effect by attempting a Fourier solution, u(x,t) = Ctic(t) exp( ikx), of 
Eq. (17.51) . We easily obtain 



ft k (t) = ft k (0) exp(-ikvt). (7.16) 

Note that is constant in time for all values of k. In other words, the amplitudes 
of the Fourier harmonics of a true solution of the advection equation remain 
constant in time — it is the phases of the harmonics which evolve. Let us now 
examine Eq. (I7.13D . It can be seen that, provided the CFL condition C < 1 
is satisfied, the magnitude of the amplification factor, |A|, is less than unity for 
all Fourier harmonics. In other words, the Lax differencing scheme causes the 
Fourier harmonics to decay in time. It is this unphysical attenuation of the Fourier 
harmonics which gives rise to the strong dispersion effect illustrated in Fig. 175 , 



Figure [76] shows a calculation made using the Lax scheme in which the CFL 
condition is violated. This calculation is identical to the one discussed previously, 
except that the time-step has been increased to 6t = 9.95 x 10 3 , yielding a CFL 
parameter, C = 2.0, which exceeds unity. It can be seen that the pulse grows in 
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3 




Figure 76: Advection of a 1-d Gaussian pulse. Numerical calculation performed using v = 1, 6t = 
9.95 x 1 -3 , and N = 200. The dotted curve shows the initial condition at t = 0.00, the short-dashed 
curve the numerical solution at t = 0.15, the long-dashed curve the numerical solution at t = 030, 
and the solid curve the numerical solution att = 0.37. 

amplitude, and eventually starts to break up due to a short-wavelength instability. 



7.4 The Crank-Nicholson Scheme 



The Crank-Nicholson implicit scheme for solving the diffusion equation (see Sect. 16.61) 
can be adapted to solve the advection equation. Thus, taking the average of the 
right-hand side of Eq. (17. 5D between the beginning and end of the time-step, we 



obtain the differencing scheme written below: 

< +1 + j «? - U-+ 1 ) = < - £ - ul, ). (7.1 7) 

A von Neumann stability analysis of the above scheme yields the following ex- 
pression for the amplification factor: 

1-i(C/2) sin(k5x) 

1 +i(C/2) sin(k6x)" U " J 

Note that |A| = 1 for all values of k, irrespective of the value of C. This implies 
that the Crank-Nicholson implicit scheme is not subject to the CFL constraint, C < 



243 



7.4 The Crank-Nicholson Scheme 



7 THE WAVE EQUATION 



1 (since there is no value of k for which |A| > 1). Moreover, there is no spurious 
decay in the Fourier harmonics of the solution (since |A| = 1). Hence, unlike 
the Lax scheme, we would not expect the Crank-Nicholson scheme to introduce 
strong numerical dispersion into the advection problem. 

Listed below is a routine which solves the 1-d advection equation via the 
Crank-Nicholson method. 

// AdvectlD.cpp 

// Function to evolve advection equation in 1-d: 

// du / dt + v du / dx = for xl <= x <= xh 

// u = at x=xl and x=xh 

// Array u assumed to be of extent N+2. 

// Now, ith element of array corresponds to 

// x_i = xl + i * dx i=0,N+l 

// Here, dx = (xh - xl) / (N+l) is grid spacing. 

// Function evolves u by single time-step. 

// C = v dt / dx, where dt is time-step. 

// Uses Crank-Nicholson scheme. 

#include <blitz/array .h> 

using namespace blitz; 

void Tridiagonal (Array<double , 1> a, Array<double , 1> b, Array<double , 1> c, 
Array<double , 1> w, Array<double , 1>& u) ; 

void AdvectlD (Array<double , 1>& u, double C) 
{ 

// Find N. Declare local arrays, 
int N = u. extent (0) - 2; 

Array<double,l> a(N+2) , b(N+2) , c(N+2), w(N+2) ; 
// Initialize tridiagonal matrix 
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for (int i = 2; i <= N; i++) a(i) = - 0.25 * C; 

for (int i = 1; i <= N; i++) b(i) = 1.; 

for (int i = 1; i <= N-l; i++) c(i) = + 0.25 * C; 

// Initialize right-hand side vector 
for (int i = 1; i <= N; i++) 

w(i) = u(i) - 0.25 * C * (u(i+l) - u(i-l)); 

// Invert tridiagonal matrix equation 
Tridiagonal (a, b, c, w, u) ; 

// Calculate i=0 and i=N+l values 
u(0) = 0.; 
u(N+l) = 0. ; 



Figure [75] shows an example calculation which uses the above routine to ad- 
vect a Gaussian pulse. The initial condition is as specified in Eq. (17.151) , and the 
calculation is performed with v = 1, 6t = 9.95 x 10 3 , and N = 200. Note that 
C = 2.0 for these parameters. It can be seen that the pulse propagates at (al- 
most) the correct speed, and maintains approximately the same shape. Clearly, 
the performance of the Crank-Nicholson scheme is vastly superior to that of the 
Lax scheme. 



7.5 Upwind Differencing 

We might be forgiven for concluding that the Crank-Nicholson scheme repre- 
sents an efficient and accurate general purpose numerical method for solving 
the advection equation. This is indeed the case, provided we restrict ourselves 
to fairly smooth wave-forms. Unfortunately, the Crank-Nicholson scheme does a 
very poor job at advecting wave-forms with sharp leading or trailing edges. This 
is illustrated in Fig. [78], which shows a calculation in which the Crank-Nicholson 
scheme is used to advect a square wave-pulse. It can be seen that spurious os- 
cillations are generated at both the leading and trailing edges of the wave-form. 
It turns out that all central difference schemes for solving the advection equation 
suffer from a similar problem. 
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Figure 77: Advection of a 1-d Gaussian pulse. Numerical calculation performed using v = ], 6t = 
9.95 x 10~ 3 , and N = 200. The solid curve shows the initial condition at I = 0, the short-dashed 
curve the numerical solution at t = 1, the long-dashed curve the numerical solution at\ = 2, and the 
dot-dashed curve the numerical solution at t = 3. 
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Figure 78: Advection of a 1-d square wave-pulse. Numerical calculation performed using v = 1, 
6t = 2.49 x ]0~ 3 , and N = 200. The dotted curve shows the initial condition at t = 0.0, whereas the 
solid curve shows the numerical solution at t = 0.1. 
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The only known way to suppress spurious oscillations at the leading and trail- 
ing edges of a sharp wave-form is to adopt a so-called upwind differencing scheme. 
In such a scheme, the spatial differences are skewed in the "upwind" direction: 
i.e., the direction from which the advecting flow emanates. Thus, the upwind 
version of the simple explicit differencing scheme (17.71) is written 



v 1 k 1 (7.19) 
ot ox 

or 

u^ 1 = uf - C « - <_! ) , (7.20) 

Note that this scheme is only first-order in space, whereas every other scheme we 
have discussed has been second-order. A von Neumann stability analysis of the 
above scheme yields 

A = 1 -C[1 -cos(k6x)] -iC sin(k6x). (7.21) 

Note that 

|A| 2 = 1 - 2 C (1 - C) [1 - cos(k6x)]. (7.22) 

It follows that |A| < 1 for all k provided that C < 1 . Thus, the upwind differenc- 
ing scheme is stable provided that the CFL condition is satisfied. Fig. [79] shows 
a calculation in which the above scheme is used to advect a square wave-pulse. 
There are now no spurious oscillations generated at the sharp edges of the wave- 
form. On the other hand, the wave-form shows evidence of dispersion. Indeed, 
the upwind differencing scheme suffers from the same type of spurious dispersion 
problem as the Lax scheme. Unfortunately, there is no known differencing scheme 
which is both non-dispersive and capable of dealing well with sharp wave-fronts. 
In fact, sophisticated codes which solve the advection (or wave) equation gener- 
ally employ an upwind scheme in regions close to sharp wave-fronts, or shocks, 
and a more accurate non-dispersive scheme elsewhere. 

Incidentally, it is easily demonstrated that the downwind differencing scheme, 

ii n+l — n n n n — n n 
ot ox 

is unconditionally unstable. 
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Figure 79: Advection of a 1-d square wave-pulse. Numerical calculation performed using v = \, 
b\ = 2.49 x 10~ 3 , and N = 200. The dotted curve shows the initial condition att = 0.0, whereas the 
solid curve shows the numerical solution at t = 0.1. 

7.6 The 1-D Wave Equation 

Consider a plane polarized electromagnetic wave propagating in vacuo along 
the z-axis. Suppose that the electric and magnetic fields take the form E = 
[E x (z, t) , 0, 0], and B = [0, (z, t) , 0]. Now, according to Maxwell's equations, 



9Ex 

at 



+ c 



+ c 



3Hy 

3z 
3z 



= 0, 



= 0, 



(7.24) 
(7.25) 



where H y = c B^, and c is the velocity of light. Note that the above equations take 
the form of two coupled advection equations. Let us find the numerical solution 
of these equations in some region < z < L which is bounded by perfectly 
conducting walls at z = and z = L. Now, both the tangential electric field and 
the normal magnetic field must be zero at a perfect conductor. It follows that 



E x (0,t)=E x (L,t)=0. 



(7.26) 
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Moreover, Eq. (I7.24D yields 



3H y (0,t) 3H y (L,t) 



0. 



(7.27) 



3z 3z 

We expect Eqs. (I7.24D and (17.251) to support wave-like solutions which bounce 
backwards and forwards between the conducting walls. 

As before, we discretize in time on the uniform grid t n = to + n 6t, for n = 
0, 1,2, Furthermore, in the z-direction, we discretize on the uniform grid 
Zi = i6z, for i = 0,1, where 6z = L/I. Adopting a Crank-Nicholson temporal 
differencing scheme similar to that discussed in Sect. 17.41 Eqs. (17.24D - 07.25D yield 



+ 



(E x ff +l -(E x ff , c fdH 
6t 

(H y )r +1 - (Hjr 



n+l 



2 V 3z 



+ 



+ 



3E, 



n+1 



+ 



: / 3>V 
2 V 3z , 

c /3E X ' 
2 I dz. 



n 



o, 



(7.28) 
(7.29) 



6t ' 2 V 3z Ji 

where (E x )^ - E x (z t ,t n ), etc. 

Adopting a Fourier approach, we write 

(Ex)? = ^t?sin(ij7t/I), 

5=0,1 

(H y )r = ^A?COS(ij7t/I), 

5=0,1 

which automatically satisfies the boundary conditions (17.261) and (17.271) . Equa- 
tions (1X281) and (1X291) yield 



(7.30) 
(7.31) 



E? +1 -E?-iD(A? +1 +An = o, 

A? +1 -A? + iD(t? +1 + tn = o, 

where D = nc 6t/(2 L). It follows that 

21D pn 1-i 2 D 2 
1 +i 2 D 2 1 1 + i 2 D 2 

2iD tr+l^A 



(7.32) 
(7.33) 



A? +1 



1 +i 2 D 2 
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n 
i > 



(7.34) 
(7.35) 
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fori = 0,1. 

Let us determine the eigenvalues A of the above linear system, assuming that 
(t^ +1 , A^ 1 ) = A At 1 ). After a little analysis, we obtain 

1 - i 2 D 2 

A 2 -2— y-^A + 1 =0. (7.36) 

For all values of i, the two roots of the above quadratic are complex, but have 
modulus unity: i.e., |A| = 1. This implies that our differencing scheme is both 
numerically stable (if |A| > 1 then the scheme would be unstable, since the as- 
sociated eigenfunction would be amplified at each time-step) and non-dispersive 
(if |A| < 1 then all Fourier harmonics would eventually decay away, and, hence, 
so would the solution) . Note that the above calculation is equivalent to von Neu- 
mann stability analysis. 

The routine listed below solves the 1-d wave equation using the Crank-Nicholson 
scheme discussed above. The routine first Fourier transforms E x and H y , takes a 
time-step using Eqs. (I7.34D and (17.351) , and then reconstructs E x and H y via an 



inverse Fourier transform. 

// WavelD.cpp 

// Function to evolve 1-d wave equation: 

// d E_x / dt + c d H_y / dz = 

// d H_y / dt + c d E_x / dz = 

// in region < z < L. 

// Boundary conditions: 

// E_x(0,t) = E_x(L,t) = 

// d H_y(0,t) / dz = d H_y(L,t) / dz = 

// Arrays Ex, Hy assumed to be of extent 1+1. 

// Now, ith elements of arrays correspond to 

// z_i = i * L / J i=0,I 

250 



7.6 The 1-D Wave Equation 



7 THE WAVE EQUATION 



// Also, D = pi c dt / (2 L), where dt is time-step. 
// Uses Crank-Nicholson scheme. 
#include <blitz/array .h> 
using namespace blitz; 

void f f t_f orward_cos (Array<double , 1> f, Array<double , 1>& F) ; 
void f f t_backward_cos (Array<double , 1> F, Array<double , 1>& f ) ; 
void f f t_f orward_sin (Array<double , 1> f, Array<double , 1>& F) ; 
void f f t_backward_sin (Array<double , 1> F, Array<double , 1>& f ) ; 

void WavelD (Array<double , 1>& Ex, Array<double , 1>& Hy, double D) 
{ 

// Find I . Declare local arrays 
int I = Ex. extent (0) - 1; 

Array<double,l> EE(I+1) , HH(I+1) , EE0(I+1), HH0(I+1); 

// Fourier transform Ex and Hy 
f f t_f orward_sin (Ex, EEO) ; 
fft_forward_cos (Hy, HHO) ; 

// Evolve EE and HH 
for (int i = 0; i <= I; i++) 
{ 

double x = double (i) * D; 
double f p = 1 . + x*x; 
double f m = 1 . - x*x; 

EE(i) =2. * x * HHO(i) + fm * EEO(i); 
EE(i) /= fp; 

HH(i) = - 2. * x * EEO(i) + fm * HHO(i) ; 
HH(i) /= fp; 

} 

// Reconstruct Ex and Hy via inverse Fourier transform 
f f t_backward_sin (EE, Ex) ; 
f f t_backward_cos (HH, Hy) ; 

> 



Figure 17711 shows an example calculation which uses the above routine to prop- 
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agate a Gaussian wave-pulse. The initial condition is 

E x (z, 0) = H y (z, 0) = exp[-100 (z - 0.5) 2 ], (7.37) 

and the calculation is performed with E = 1,c = 1,6t = 5x 10 3 , and N = 100. 
It can be seen that the pulse moves to the right, reflects off the conducting wall 
at z = 1 , and then moves to the left. Note, that E x = +H y when the wave is far 
from the conducting walls and moving to the right, whereas E x = — H y when the 
wave is far from the conducting walls and moving to the left. 



Figure 17.241 shows a second example calculation performed with the same 



parameters as the first. In this calculation, the pulse is allowed to reflect off 
the conducting walls ten times before returning to its initial position. Note that 
the pulse amplitude and shape remain constant to a very good approximation 
during this process. The fact that the pulse returns almost exactly to its initial 
position after ten time periods have elapsed (i.e., at t = 10) demonstrates that it 
is propagating at the correct speed (i.e., c = 1). 

Figure [82] shows a third example calculation which uses the above listed rou- 
tine to propagate a square wave-pulse. Note that the routine does a very poor 
job, since spurious oscillations are generated at the sharp leading and trailing 
edges of the wave-form. As discussed in Sect. 17.51 such oscillations can only be 



suppressed by adopting an upwind differencing scheme, which in this case means 
that the spatial differences must be skewed in the direction from which the wave 
is propagating. Unfortunately, simple explicit upwind schemes are subject to the 
CFL constraint, 

6z 

6t<— (7.38) 
c 

and also tend to be highly dispersive. 



7.7 The 2-D Resonant Cavity 



Figure [83] shows a 2-d resonant cavity consisting of a hollow, rectangular, per- 
fectly conducting channel of dimensions L x x L y . Suppose that the walls of the 
channel are aligned along the x- and u-axes. We shall excite this cavity in a 
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Figure 80: Propagation of a 1-d Gaussian wave-pulse. Numerical calculation performed using c = 1, 
St = 5 x 10~ 3 , and N = 100. The solid curves show whereas the dashed curves show H y . The 
top-left, top-right, middle-left, middle-right, bottom-left, and bottom-right panels show the solution 
att = 0.0, 0.2, OA, 0.6, 0.8, and 1 .0, respectively. 
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Figure 81: Propagation of a 1-d Gaussian wave-pulse. Numerical calculation performed using c = \, 
5t = 5 x 10~ 3 , and N = 100. The solid curve shows E x at t = 0., the dashed curve shows E x at 
t = 1 0., and the dotted curve (obscured by the dashed curve) shows B v at t = 1 0. 



H 0.5 - 




Figure 82: Propagation of a 1-d square wave-pulse. Numerical calculation performed using c = 1, 
5t = 5 x 10~ 3 , and N = 100. The dotted curve shows E x at t = 0.0, and the solid curve shows E x at 
t = 0.1. 
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conducting walls 



^x 



X 



Figure 83: A 2-d resonant cavity. 



rather artificial manner by imposing a z-directed alternating current pattern of 
frequency f, which has the same spatial structure as the mode in which we are 
interested. Let us calculate the electric and magnetic field patterns excited within 
the cavity by such a current pattern. 

The electric and magnetic fields within the cavity can be written E = [0,0, 
E 2 (x,y,t)], andB = [B x (x,y, t), B y (x,u, t), 0], respectively. It follows from Maxwell's 
equations that 

9H aE z 

+ c = 0, 



at 

3H 



9y 

3E, 



at 



v +c^ = 0, 



9x 



(7.39) 
(7.40) 
(7.41) 
-(x c 2 j z . Note 

that the above system of equations takes the form of three coupled advection 
equations with a source term. The boundary conditions are that the tangential 
electric field and the normal magnetic field must be zero at the conducting walls. 



9E 3H 3H X 

+ c — - + c = J z , 

3t 3x dy 



where c is the velocity of light, H x = c B x , H v 



c B y , and J 2 
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It follows that 

E z = (7.42) 
at all the walls (which are located at x = 0, L x and y = 0, L v ), 

H x = ^ = (7.43) 
ox 

at x = 0, L x , and 

H y = ^ = (7.44) 

at y — 0, L y . Finally, the normalized current pattern associated with the (m, n) 
mode takes the form 

J z (x,-y,t) = Jo sin(m,7rx/L x ) sin(u7t-y/L y ) sin(27if t). (7.45) 

As usual, we discretize in time on the uniform grid t n = to + n St, for n = 
0,1,2, Furthermore, in the x-direction, we discretize on the uniform grid 
Xi = i6x, for i = 0, 1, where 6x = L x /I. Finally, in the y -direction, we discretize 
on the uniform grid Uj = j by, for j = 0, J, where by = Ly/J. Adopting a Crank- 
Nicholson temporal differencing scheme similar to that discussed in Sects. 17.4 



-Z 



and[7Sl Eqs. (17391) - (1X4TT ) yield 

c /3E z \ n+1 + c /3E z \ n _ 
2 V 3x 2 V 3x A,j 

c ^3H^ n+1 + c ^3H^ n 



St 








St 











St 2 V 3x Aj 2 V 3x Aj 

where (H x )^ = H x (xi,Uj, t n ), etc. 

Adopting a Fourier approach, we write 

j'=oj 

( E *)5j = L ^j'Sinfii'Tt/I) sinCjj'Tr/J), (7.49) 

i'=0,I 
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j'=OJ 

( H *)?j = ^ sinfii'Ti/I) cosfjj'Ti/J), (7.50) 

i'=0,I 

j'=o,J 

( H ^i = ^^'COsliiV/DsinOj'Tr/J), (7.51) 

i'=0,I 

j'=o,J 

(J 2 )^ = X.ftj,sin(UVl)sin(jj'7r/J) (7.52) 

i'=0,I 

which automatically satisfies the boundary conditions (I7.42D — (I7.44D . Equations (17.46D 
(1Z48D yield 

^ 1 -^.+jD,(^ 1 +ty = 0, (7.53) 

^ 1 -^ + iD x (tJ 1 +^) = 0, (7.54) 

f- +1 -t--iDJ^V^)-jD j (^ +1 +^ j ) = 5tT&, (7.55) 

for i = 0,1 and j = 0, J, where D x = 7tc6t/(2L x ) and D y = 7tc5t/(2L y ). It 
follows that 

(1-i 2 D x -j 2 D^^+2jD,^ j + 2iP x % + 6t^ 



fn+1 



^ = ^-iDjE^+E^. 



(7.56) 
(7.57) 
(7.58) 



The routine listed below solves the 2-d wave equation in a resonant cavity 
using the Crank-Nicholson scheme discussed above. The routine first Fourier 
transforms H x , H v , E z , and J z in both the x- and y -directions, takes a time-step 
using Eqs. (17.56D — (17.581) , and then reconstructs H x , H y , and E z via an double 
inverse Fourier transform. 



/ / Wave2D . cpp 

// Function to evolve 2-d wave equation: 
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// d H_x / dt + c d E_z / dy = 
// d H_y / dt + c d E_z / dx = 

// d E_z / dt + c d H_y / dx + c d H_x / dy = J_z 
// in region < x < L_x and < y < L_y 
// Boundary conditions: 

// E_z(0, y) = E_z(L_x, y) = E_z(x, 0) = E_z(x, L_y) = 

// H_x(0, y) = H_x(L_x, y) = d H_y(0, y) / dx = d H_y(L_x, y) / dx = 

// H_y(x, 0) = H_y(x, L_y) = d H_x(x, 0) / dy = d H_x(x, L_y) / dy = 

// Matrices Hx, Hy, Ez, Jz assumed to be of extent 1+1 , J+l. 
// Now, (i,j)th elements of matrices correspond to 

// x_i = i * dx i=0,I 
// y-j = j * dy j=0,J 

// Here, dx = L_x / I is grid spacing in x-direction, 

// and dy = L_y / J is grid spacing in x-direction. 

// Now, Dx = pi c dt / (2 L_x) and Dy = pi c dt / (2 L_y) , 

// where dt is time-step. 

// Uses Crank-Nicholson scheme. 

#include <blitz/array .h> 

using namespace blitz; 

void f f t_f orward_cos (Array<double , 1> f, Array<double , 1>& F) ; 
void f f t_backward_cos (Array<double , 1> F, Array<double , 1>& f ) ; 
void f f t_f orward_sin (Array<double , 1> f, Array<double , 1>& F) ; 
void f f t_backward_sin (Array<double , 1> F, Array<double , 1>& f ) ; 

void Wave2D (Array<double,2>& Hx, Array<double ,2>& Hy, Array<double , 2>& Ez, 

Array<double,2> Jz, double Dx, double Dy, double dt) 

{ 
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// Find I and J. Declare local arrays 

int I = Hx. extent (0) - 1; 

int J = Hx. extent (1) - 1; 

Array<double , 2> X(I+1, J+l) , XX(I+1, J+l) , XXX(I+1, J+l) 

Array<double , 2> Y(I+1, J+l), YY(I+1, J+l), YYY(I+1, J+l) 

Array<double , 2> E(I+1, J+l), EE(I+1, J+l), EEE(I+1, J+l) 

Array<double , 2> K(I+1, J+l), KK(I+1, J+l); 

// Fourier transform solution in x-direction 
for (int j =0; j <= J; j++) 
{ 

Array<double,l> In(I+l) , 0ut(I+l); 
// Fourier transform Hx 

for (int i = 0; i <= I; i++) In(i) = Hx(i, j); 
f f t_f orward_sin (In, Out); 

for (int i = 0; i <= I; i++) X(i, j) = Out(i); 
// Fourier transform Hy 

for (int i = 0; i <= I; i++) In(i) = Hy(i, j); 
f f t_f orward_cos (In, Out); 

for (int i = 0; i <= I; i++) Y(i, j) = 0ut(i); 
// Fourier transform Ez 

for (int i = 0; i <= I; i++) In(i) = Ez(i, j); 
f f t_f orward_sin (In, Out) ; 

for (int i = 0; i <= I; i++) E(i, j) = 0ut(i); 
// Fourier transform Jz 

for (int i = 0; i <= I; i++) In(i) = dt * Jz(i, j); 
f f t_f orward_sin (In, Out) ; 

for (int i = 0; i <= I; i++) K(i, j) = Out(i); 

} 

// Fourier transform solution in y-direction 
for (int i = 0; i <= I; i++) 
{ 

Array<double,l> In(J+l) , Out (J+l); 
// Fourier transform Hx 

for (int j = 0; j <= J; j++) In(j) = X(i, j); 
f f t_f orward_cos (In, Out) ; 

for (int j = 0; j <= J; j++) XX(i, j) = Out(j); 
// Fourier transform Hy 
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for (int j = 0; j <= J; In(j) = Y(i, j); 

f f t_f orward_sin (In, Out) ; 

for (int j = 0; j <= J; YY(i, j) = Out(j); 

// Fourier transform Ez 

for (int j = 0; j <= J; In(j) = E(i, j); 

f f t_f orward_sin (In, Out) ; 

for (int j = 0; j <= J; j++) EE(i, j) = Out(j); 
// Fourier transform Jz 

for (int j = 0; j <= J; j++) In(j) = K(i, j); 
f f t_f orward_sin (In, Out); 

for (int j = 0; j <= J; j++) KK(i, j) = Out(j); 

} 

// Evolve XX, YY, and EE 
for (int i = 0; i <= I; 

for (int j =0; j <= J; j++) 
{ 

double x = double (i) * Dx; 
double y = double (j) * Dy; 
double f p = 1 . + x*x + y*y; 
double f m = 1 . - x*x - y*y; 

EEE ( i , j) = fm * EE(i, j) + 2. * y * XX(i, j) + 

2. * x * YY(i, j) + KK(i, j) ; 
EEE ( i , j) /= fp; 

XXX(i, j) = XX(i, j) - y * (EEE(i, j) + EE(i, j)); 
YYY ( i , j) = YY(i, j) - x * (EEE(i, j) + EE(i, j)); 

> 

// Reconstruct solution via inverse Fourier transform in y-direction 
for (int i = 0; i <= I; 
{ 

Array<double,l> In(J+l) , Out(J+l); 
// Reconstruct Hx 

for (int j = 0; j <= J; j++) In(j) = XXX(i, j); 
f f t_backward_cos (In, Out) ; 

for (int j = 0; j <= J; j++) X(i, j) = Out(j); 
// Reconstruct Hy 

for (int j =0; j <= J ; j++) In(j) = YYY ( i , j); 
f f t_backward_sin (In, Out) ; 

for (int j = 0; j <= J; j++) Y(i, j) = Out(j); 
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// Reconstruct Ez 

for (int j =0; j <= J; In(j) = EEE ( i , j); 

f f t_backward_sin (In, Out) ; 

for (int j = 0; j <= J; E(i, j) = Out(j); 

} 

// Reconstruct solution via inverse Fourier transform in x-direction 
for (int j =0; j <= J; j++) 
{ 

Array<double, 1> In(I+l) , 0ut(I+l); 
// Reconstruct Hx 

for (int i = 0; i <= I; i++) In(i) = X(i, j); 
f f t_backward_sin (In, Out) ; 

for (int i = 0; i <= I; i++) Hx(i, j) = 0ut(i); 
// Reconstruct Hy 

for (int i = 0; i <= I; i++) In(i) = Y(i, j); 
f f t_backward_cos (In, Out) ; 

for (int i = 0; i <= I; i++) Hy(i, j) = 0ut(i); 
// Reconstruct Ez 

for (int i = 0; i <= I; i++) In(i) = E(i, j); 
f f t_backward_sin (In, Out) ; 

for (int i = 0; i <= I; i++) Ez(i, j) = 0ut(i); 

} 



The numerical calculations discussed below were performed using the above 
routine. The electromagnetic fields H x , H y , and E z were all initialized to zero 
everywhere at t = 0. Figure [841 shows the maximum amplitude of E z versus the 
frequency, f, for an m = 1 /n = 1 driving current distribution. It can be seen that 
there is a clear resonance at f ~ 0.7. 



Figures [85] and [86] illustrate the typical time variation of E z , H x , and H y for 



a non-resonant and a resonant case, respectively. For the non-resonant case, the 
traces take the form of interference patterns between the directly driven response, 
which oscillates at the driving frequency f, and the transient response, which 
oscillates at the natural frequency f o of the cavity. Note that the transients never 
decay, since there is no dissipation in the present problem. Incidentally, it is easily 
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Figure 84: Electromagnetic waves in a 2-d resonant cavity. The maximum amplitude of E z (x = 
0.5, y = 0.5) between t = and t = 20 versus the driving frequency, f. Numerical calculation 
performed using m = 1, n = 1, L x = 1, Ly = 1, c = 1, I = J = 32, and 8t = 10~ 2 . 

demonstrated that 



1 + t 1 rr- (7.59) 



Hence, it follows that fo = l/y/2 = 0.707] for n = m = L x = L y = c = 1, 
which corresponds very well to the resonant frequency found in Fig. (84J. For the 
resonant case, the traces take the form of waves of ever increasing amplitude 
which oscillate at the natural frequency fo. 



Finally, Figs. [86] and [87] illustrate the spatial variation of the electromagnetic 



fields driven within the cavity when m = 1 and n = 1 
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Figure 85: Electromagnetic waves in a 2-d resonant cavity. Time traces o/E z (x = 0.5, y = 0.5) (solid 
curve), H x (x = 0.5, y = 0.0) (dashed curve), and H y (x = 0.0, y = 0.5) (dotted curve — obscured 
by dashed curve). Numerical calculation performed using m = 1, n = 1, L x = 1, Ly = 1, c = 1, 
I = J = 32, St = 10- 2 , and f = 0.6. 




Figure 86: Electromagnetic waves in a 2-d resonant cavity. Time traces o/E z (x = 0.5, y = 0.5) (solid 
curve), H x (x = 0.5, y = 0.0) (dashed curve), and H y (x = 0.0, y = 0.5) (dotted curve — obscured 
by dashed curve). Numerical calculation performed using m = 1, n = 1, L x = 1, L y = 1, c = 1, 
I = J = 32, St = ]Q- 2 , and f = 0.7071. 
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Figure 87: Electromagnetic waves in a 2-d resonant cavity. Spatial variation o/E z (solid curve) and 
H y (dashed curve) in the x-direction at t = 20 and y = 0.5. Numerical calculation performed using 
m. = 1, n = 1, L x = 1, L y = 1, c = 1, I = J = 32, 6t = 10~ 2 , and f = 0.7071. 



1 I — 1 1 — 1 — I — 1 1 1 — I — 1 — ! 1 — | — 1 — 1 — 1 — | — 1 1 r 




0.2 0.4 0.6 0.8 1 

y 

Figure 88: Electromagnetic waves in a 2-d resonant cavity. Spatial variation o/E z (solid curve) and 
H x (dashed curve) in the y-direction att = 20 and x = 0.5. Numerical calculation performed using 
m = 1, u = 1, L x = 1, L y = 1, c = 1, I = J = 32, 5t = 10 -2 , and f = 0.7071. 
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8.1 Introduction 

Consider an unmagnetized, uniform, 1 -dimensional plasma consisting of N elec- 
trons and N unit-charged ions. Now, ions are much more massive than electrons. 
Hence, on short time-scales, we can treat the ions as a static neutralizing back- 
ground, and only consider the motion of the electrons. Let r t be the x-coordinate 
of the ith electron. The equations of motion of the ith electron are written: 



dr t 
dt 

dvi eE(ri) 



= v t , (8.1) 
¥(rA 

(8-2) 
dt m e 

where e > is the magnitude of the electron charge, m e the electron mass, and 
E(x) the x-component of the electric field-strength at position x. Now, the electric 
field-strength can be expressed in terms of an electric potential: 

E(x) = — . (8.3) 

dx 

Furthermore, from the Poisson-Maxwell equation, we have 

, 7 {no-n(xj), (8.4) 

dx z eo 

where eo is the permittivity of free-space, n(x) the electron number density (i.e., 
n(x) dx is the number of electrons in the interval x to x+ dx), and no the uniform 
ion number density. Of course, the average value of n(x) is equal to no, since 
there are equal numbers of ions and electrons. 

Let us consider an initial electron distribution function consisting of two counter- 
propagating Maxwellian beams of mean speed v b and thermal spread v th : i.e., 

f(x,v) = ^ {—^ e -^- v ^ 2 < + -= ^ e - (v+ ^ 2 / 2v 4 . (8.5) 
2 [V2nv th V2nv th J 
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Here, f (x, v) dx dv is the number of electrons between x and x+ dx with velocities 
in the range v to v + dv. Of course, n(x) = J 00 ^ f (x, v) dv. The beam temperature 
T is related to the thermal velocity via v th . = J\cb T/m e , where ke is the Boltzmann 
constant. It is well-known that if v b is significantly larger than v th . then the above 
distribution is unstable to a plasma instability called the two-stream instability^! 
Let us investigate this instability numerically. 



8.2 Normalization Scheme 

It is convenient to normalize time with respect to uo v \ where 

2 n e 2 



o>; = (8.6) 

p e m e 

is the so-called plasma frequency: i.e., the typical frequency of electrostatic elec- 
tron oscillations. Likewise it is convenient to normalize length with respect to the 
so-called Debye length: 

Ad = — , 

which is the length-scale above which the electrons exhibit collective (i.e., plasma- 
like) effects, instead of acting like individual particles. 



Our normalized equations take the form: 

v t> (8.7) 



dxi 

dvi 
lit 



-E(xO, (8.8) 



E(x) = -^1, (8.9) 
dx 

^ = (8.10) 

ax 1 no 
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T.H. Stix, The theory of plasma waves, 1st Ed. (McGraw-Hill, New York NY, 1962). 
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whereas our initial distribution function becomes 

f(x,v) = ^ (_L e (v Vb)2 / 2 + ^Le (V+Vb)2 / 2 1 . (8.11) 
2 [y/2n V2n J 

Note that v th . = 1 in normalized units. 

Let us solve the above system of equations in the domain < x < L. Fur- 
thermore, for the sake of simplicity, let us adopt periodic boundary conditions: 
i.e., let us identify the left and right boundaries of our solution domain. It fol- 
lows that n(0) = n(L), c()(0) = c()(L), and E(0) = E(L). Moreover, any electron 
which crosses the right boundary of the solution domain must reappear at the 
left boundary with the same velocity, and vice versa. 



8.3 Solution of Electron Equations of Motion 



We can solve the electron equations of motion, (18. 7D and (18.81) . as a set of 2N 



coupled first-order ODEs using the RK4 methods discussed in Sect. [31 However, 
in order to evaluate the right-hand sides of these equations we need to know 
the electric field E(x) at each time-step. We can achieve this by solving Poisson's 
equation, (18.101) . every time-step. However, in order to determine the source 



term for this equation we need to calculate the electron number density n(x), 
which is, of course, a function of the instantaneous electron locations. 



8.4 Evaluation of Electron Number Density 

In order to obtain the electron number density u(x) from the electron coordinates 
Ti we adopt a so-called particle-in-cell (PIC) approach. Let us define a set of J 
equally spaced spatial grid-points located at coordinates 

Xj=j6x, (8.12) 
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for j = 0, J — 1, where Sx = L/J. Let rij = n(xj). Suppose that the ith electron lies 
between the jth and (j + 1 )th grid-points: i.e., Xj < n < Xj+i. We let 




n i n J + I — IT I / 6x > ( 8 - 13 ) 

and 



n-Xj 



n j+1 -> n j+ i + — / 6x. (8.14) 

Thus, Uj 6x increases by 1 if the electron is at the jth grid-point, Uj+i 6x increases 
by 1 if the electron is at the + 1 )th grid-point, and nj 6x and nj+i 6x both increase 
by 1/2 if the electron is halfway between the two grid-points, etc. Performing a 
similar assignment for each electron in turn allows us to build up the Uj from the 
electron coordinates (assuming that all the Uj are initialized to zero at the start 
of this process) . 



8.5 Solution of Poisson's Equation 

Consider the solution of Poisson's equation 

d 2 cMx) 



dx 2 



= p(x), (8.15) 



where p(x) = n(x)/no — 1. Note that no = N/L in normalized units. Let = 
4>(xj) and pj = p(xj). We can write 

^ = Y. $Y* m ' 2n/ \ (8.16) 
j'=o,J-i 

Pj = Y. ft'e 1 "' 27171 , (8-17) 
j'=oj-i 

which automatically satisfies the periodic boundary conditions (J)j = 4>o and pj = 
po. Note that = 0, since Jon(x) dx = no. The other ^ are obtained from 

h = \ Z Pre ijj ' 27t/I , (8.18) 
j j'=oj-i 
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for j = 1 , J — 1 . The Fourier transformed version of Poisson's equation yields 

$o = (8.19) 

and 

$j = - A (8.20) 

) l K 

for j = 1 , J/2, where k = 2n/L. Finally, 

$j = (8.21) 



for j = J/2 + 1 to J — 1, which ensures that the c()j remain real. The discretized 
version of Eq. (18. 9D is 

= *H - fej . (8 . 22) 



Of course, j = and j = J — 1 are special cases which can be resolved using 
the periodic boundary conditions. Finally, suppose that the coordinate of the ith 
electron lies between the jth and (j + 1 )th grid-points: i.e., Xj < r\ < Xj+i . We can 
then use linear interpolation to evaluate the electric field seen by the ith electron: 



E(n) = ( + ] E j+1 . (8.23) 



8.6 An example 1-D PIC Code 

The following code is an implementation of the ideas developed above. 

The main function reads in the calculation parameters, checks that they are 
sensible, initializes the electron coordinates, and then evolves the electron equa- 
tions of motion from t = to some specified t max , using a fixed step RK4 routine 
with some specified time-step 6t. Information on the electron phase-space coor- 
dinates and the electric field is periodically written to various data-files. 

// 1-d PIC code to solve plasma two-stream instability problem. 
#include <stdlib.h> 
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#include <stdio.h> 
#include <math.h> 
#include <t ime . h> 
#include <blitz/array .h> 
#include <fftw.h> 

using namespace blitz; 

void Output (char* fnl, char* fn2, double t, 

Array<double , 1> r, Array<double , 1> v) ; 
void Density (Array<double , 1> r, Array<double , 1>& n) ; 
void Electric (Array<double , 1> phi, Array<double , 1>& E) ; 
void PoissonlD (Array<double , 1>& u, Array<double , 1> v, double kappa); 
void rk4_fixed (doublefe x, Array<double , 1>& y, 

void (*rhs_eval) (double , Array<double , 1> , Array<double , 1>&) , 
double h) ; 

void rhs_eval (double t, Array<double , 1> y, Array<double , 1>& dydt) ; 
void Load (Array<double , 1> r, Array<double , 1> v, Array<double , 1>& y) ; 
void UnLoad (Array<double , 1> y, Array<double , 1>& r, Array<double , 1>& v) ; 
double distribution (double vb) ; 

double L; int N, J; 

int main() 
{ 

// Parameters 

L; // Domain of solution <= x <= L (in Debye lengths) 

N; // Number of electrons 

J; // Number of grid points 

double vb; // Beam velocity 

double dt ; // Time-step (in inverse plasma frequencies) 

double tmax; // Simulation run from t = 0. to t = tmax 

// Get parameters 
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printf ("Please input N: "); scanf ('"/.d", &N) ; 
printf ("Please input vb: "); scanf ("°/ lf", &vb) ; 
printf ("Please input L: "); scanf ("•/.If", &L) ; 
printf ("Please input J: "); scanf ("%d", &J) ; 
printf ("Please input dt : "); scanf ("°/ lf", &dt) ; 
printf ("Please input tmax: "); scanf ("°/ lf", fetmax) ; 
int skip = int (tmax / dt) / 10; 

if ((N < 1) II (J < 2) || (L <= 0.) I I (vb <= 0.) 
II (dt <= 0.) II (tmax <= 0.) II (skip < 1)) 

{ 

printf ("Error - invalid input parameters\n") ; 
exit (1) ; 

} 

// Set names of output files 
char* phase [11]; char* data [11] ; 

phase [0] = "phaseO . out "; phase [1] = " phase 1 . out "; phase [2] = "phase2.out" 
phase [3] = "phase3 . out " ; phase [4] = "phase4.out" ; phase [5] = "phase5.out" 
phase [6] = "phase6 . out " ; phase [7] = "phase7.out" ; phase [8] = "phase8.out" 
phase [9] = "phase9 . out "; phase [10] = "phaselO. out "; data [0] = "dataO.out" 
data[l] = "datal.out"; data [2] = "data2.out"; data [3] = "data3.out"; 
data [4] = "data4.out"; data [5] = "data5.out"; data [6] = "data6.out"; 
data [7] = "data7.out"; data [8] = "data8.out"; data [9] = "data9.out"; 
data [10] = "datalO.out" ; 

// Initialize solution 
double t = . ; 

int seed = time (NULL) ; srand (seed) ; 
Array<double , 1> r(N), v(N) ; 
for (int i = 0; i < N; i++) 
{ 

r(i) = L * double (rand ()) / double (RAND_MAX) ; 
v(i) = distribution (vb) ; 

} 
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Output (phase [0] , data[0] , t, r, v) ; 

// Evolve solution 
Array<double , 1> y(2*N) ; 
Load (r, v, y) ; 
for (int k = 1; k <= 10; k++) 
{ 

for (int kk = 0; kk < skip; kk++) 
{ 

// Take time-step 

rk4_fixed (t, y, rhs_eval, dt) ; 

// Make sure all coordinates in range to L. 
for (int i = 0; i < N; i++) 
{ 

if (y(i) < 0.) y(i) += L; 
if (y(i) > L) y(i) -= L; 

} 

printf ("t = °/oll.4e\n", t) ; 

} 

printf ("Plot °/ 3d\n\ k) ; 

// Output data 
UnLoad (y , r , v) ; 

Output (phase [k] , data[k] , t, r, v) ; 

} 

return 0; 



The following routine outputs the simulation data to various data-files. 
// Write data to output files 
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void Output (char* fnl, char* fn2, double t, 
Array<double , 1> r, Array<double , 1> v) 

{ 

// Write phase-space data 
FILE* file = fopen (fnl, "w"); 
for (int i = 0; i < N; i++) 

fprintf (file, M %e %e\n\ r(i) , v(i)); 
f close (file); 

// Write electric field data 
Array<double, 1> ne(J), n(J) , phi (J), E(J) ; 
Density (r, ne) ; 
for (int j = 0; j < J; j++) 

n(j) = double (J) * ne(j) / double (N) - 1.; 
double kappa = 2. * M_PI / L; 
PoissonlD (phi, n, kappa); 
Electric (phi, E) ; 

file = fopen (fn2, V); 
for (int j = 0; j < J; j++) 
{ 

double x = double (j) * L / double ( J) ; 

fprintf (file, " / e °/„e °/„e °/„e\n" , x, ne(j), n(j), E(j)); 

} 

double x = L; 

fprintf (file, ,,0 /„e °/„e °/„e 7„e\n M , x, ne(0), n(0) , E(0)); 
f close (file); 



The following routine returns a random velocity distributed on a double Maxwellian 
distribution function corresponding to two counter-streaming beams. The algo- 
rithm used to achieve this is called the rejection method, and will be discussed 
later in this course. 
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// Function to distribute electron velocities randomly so as 
//to generate two counter propagating warm beams of thermal 
// velocities unity and mean velocities +/- vb. 
// Uses rejection method. 

double distribution (double vb) 
{ 

// Initialize random number generator 
static int flag = 0; 
if (flag == 0) 
{ 

int seed = time (NULL) ; 
srand (seed) ; 
flag = 1; 

} 

// Generate random v value 

double fmax = 0.5 * (1. + exp (-2. * vb * vb)); 
double vmin = - 5. * vb; 
double vmax = + 5. * vb; 

double v = vmin + (vmax - vmin) * double (rand ()) / double (RAND_MAX) ; 
// Accept/reject value 

double f = 0.5 * (exp (-(v - vb) * (v - vb) / 2.) + 

exp (-(v + vb) * (v + vb) / 2.)); 
double x = fmax * double (rand ()) / double (RAND_MAX) ; 
if (x > f) return distribution (vb) ; 
else return v; 



The routine below evaluates the electron number density on an evenly spaced 
mesh given the instantaneous electron coordinates. 

// Evaluates electron number density n(0:J-l) from 
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// array r(0:N-l) of electron coordinates. 

void Density (Array<double , 1> r, Array<double , 1>& n) 
{ 

// Initialize 

double dx = L / double (J) ; 
n = 0. ; 

// Evaluate number density, 
for (int i = 0; i < N; i++) 
{ 

int j = int (r(i) / dx) ; 

double y = r(i) / dx - double (j); 

n(j) += (1. - y) / dx; 

if (j+1 == J) n(0) += y / dx; 

else n(j+l) += y / dx; 

} 

} 

The following functions are wrapper routines for using the fftw library with 
periodic functions. 

// Functions to calculate Fourier transforms of real data 

// using fftw Fast-Fourier-Transform routine. 

// Input/ouput arrays are assumed to be of extent J. 

// Calculates Fourier transform of array f in arrays Fr and Fi 
void fft_forward (Array<double , l>f , Array<double , l>&Fr , 
Array<double , 1>& Fi) 

{ 

fftw_complex f f [ J] , FF[J]; 
// Load data 

for (int j = 0; j < J; j++) 
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{ 

c_re (ff[j]) = f(j); c_im (ff[j]) = 0.; 

} 

// Call fftw routine 

fftw_plan p = fftw_create_plan (J, FFTW_F0RWARD , FFTW_ESTIMATE) ; 
fftw_one (p, ff, FF); 
f ftw_destroy_plan (p) ; 

// Unload data 
for (int j = 0; j < J; j++) 
{ 

Fr(j) = c_re (FF [j] ) ; Fi(j) = c_im (FF [j] ) ; 

} 

// Normalize data 
Fr /= double ( J) ; 
Fi /= double (J) ; 

} 

// Calculates inverse Fourier transform of arrays Fr and Fi in array f 
void fft_backward (Array<double , 1> Fr, Array<double , 1> Fi , 
Array<double, 1>& f) 

{ 

fftw_complex f f [ J] , FF[J]; 
// Load data 

for (int j = 0; j < J; j++) 
{ 

c_re (FF [j] ) = Fr(j); c_im (FF [j] ) = Fi(j); 

} 

// Call fftw routine 

fftw_plan p = fftw_create_plan (J, FFTW_BACKWARD , FFTW_ESTIMATE) ; 
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fftw_one (p, FF, f f ) ; 
f ftw_destroy_plan (p) ; 

// Unload data 
for (int j = 0; j < J; j++) 
f(j) = c_re (ff[j]); 

} 

The following routine solves Poisson's equation in 1-D to find the instanta- 
neous electric potential on a uniform grid. 

// Solves 1-d Poisson equation: 

// d~u / dx~2 = v for <= x <= L 

// Periodic boundary conditions: 

// u(x + L) = u(x), v(x + L) = v(x) 

// Arrays u and v assumed to be of length J. 

// Now, jth grid point corresponds to 

// x_j = j dx for j = 0,J-1 

// where dx = L / J. 

// Also, 

// kappa = 2 pi / L 

void PoissonlD (Array<double , 1>& u, Array<double , 1> v, double kappa) 
{ 

// Declare local arrays. 

Array<double,l> Vr(J) , Vi(J), Ur(J) , Ui(J); 

// Fourier transform source term 
fft_forward (v, Vr, Vi) ; 

// Calculate Fourier transform of u 
Ur(0) = Ui(0) = 0. ; 
for (int j = 1; j <= J/2; j++) 
{ 
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Ur(j) = - Vr(j) / double (j * j) / kappa / kappa; 
Ui(j) = - Vi(j) / double (j * j) / kappa / kappa; 

} 

for (int j = J/2; j < J; j++) 
{ 

Ur(j) = Ur(J-j); 
Ui(j) = - Ui(J-j); 

} 

// Inverse Fourier transform to obtain u 
fft_backward (Ur, Ui, u) ; 

} 

The following function evaluates the electric field on a uniform grid from the 
electric potential. 

// Calculate electric field from potential 

void Electric (Array<double , 1> phi, Array<double , 1>& E) 
{ 

double dx = L / double (J) ; 

for (int j = 1; j < J-l; j++) 

E(j) = (phi(j-l) - phi(j+l)) / 2. / dx; 
E(0) = (phi(J-l) - phi(l)) / 2. / dx; 
E(J-l) = (phi(J-2) - phi(O)) / 2. / dx; 

} 

The following routine is the right-hand side routine for the electron equations 
of motion. Is is designed to be used with the fixed-step RK4 solver described 
earlier in this course. 

// Electron equations of motion: 
// y(0:N-l) = r_i 
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// y(N:2N-l) = dr_i/dt 

void rhs_eval (double t , Array<double , 1> y, Array<double , 1>& dydt) 
{ 

// Declare local arrays 

Array<double,l> r(N) , v(N) , rdot(N), vdot(N), rO(N) ; 
Array<double , 1> ne(J), rho(J), phi (J) , E(J); 

// Unload data from y 
UnLoad (y , r , v) ; 

// Make sure all coordinates in range to L 
rO = r; 

for (int i = 0; i < N; i++) 
{ 

if (r0(i) < 0.) r0(i) += L; 
if (r0(i) > L) r0(i) -= L; 

} 

// Calculate electron number density 
Density (rO, ne) ; 

// Solve Poisson's equation 
double nO = double (N) / L; 
for (int j = 0; j < J; j++) 
rho(j) = ne(j) / nO - 1 . ; 
double kappa = 2. * M_PI / L; 
PoissonlD (phi, rho, kappa); 

// Calculate electric field 
Electric (phi, E) ; 

// Equations of motion 

for (int i = 0; i < N; i++) 
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{ 

double dx = L / double (J) ; 
int j = int (rO(i) / dx) ; 
double y = rO(i) / dx - double (j); 

double Efield; 
if (j+1 == J) 

Efield = E(j) * (1. 
else 

Efield = E(j) * (1. 

rdot (i) = v(i) ; 
vdot(i) = - Efield; 

} 

// Load data into dydt 
Load (rdot, vdot , dydt); 

} 

The following functions load and unload the electron phase-space coordinates 
into the solution vector y used by the RK4 routine. 

// Load particle coordinates into solution vector 

void Load (Array<double , 1> r, Array<double , 1> v, Array<double , 1>& y) 
{ 

for (int i = 0; i < N; i++) 
{ 

y(i) = r(i); 
y(N+i) = v(i); 

} 

} 

// Unload particle coordinates from solution vector 



- y) + E(0) * y; 

- y) + E(i+1) * y; 
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X X 

Figure 89: The electron phase-space distribution evaluated at various times for a 1 -dimensional two- 
stream instability calculation performed with N = 20000, J = 1000, L = 100, v b = 3, and St = 0.1. 



void UnLoad (Array<double , 1> y, Array<double , 1>& r, Array<double , 1>& v) 
{ 

for (int i = 0; i < N; i++) 
{ 

r(i) = y(i); 
v(i) = y(N+i); 

} 

} 



8.7 Results 



Figures [891 and [901 show the electron phase-space distributions evaluated at equally 
spaced times for a two-stream instability calculation performed with 2 x 1 4 elec- 
trons. It can be seen that the distribution initially takes the form of two uni- 
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Figure 90: The electron phase-space distribution evaluated at various times for a 1 -dimensional two- 
stream instability calculation performed with N = 20000, J = 1000, L = 100, v b = 3, and 5t = 0.1. 



form bands, corresponding to two counter-streaming electron beams. However, 
as time progresses, the bands spontaneously develop structure which grows in 
magnitude and eventually converts the phase-space distribution into a set of con- 
nected vortices. In this final state, the electrons are basically bouncing backwards 
and forwards in a quasi-periodic electric potential generated by non-uniformities 
in the electron density. In other words, the instability effectively destroys the two 
beams. For this reason, the two-stream instability is of major concern in particle 
accelerators, which often consist of counter-propagating charged particle beams. 



8.8 Discussion 



Obviously, the ideas discussed above could be generalized in a fairly straight- 
forward manner to deal with the evolution of two and three-dimensional charged 
particle distributions. PIC codes have the advantage that they are reasonably 
straight-forward to write. Unfortunately, PIC codes also have a number of disad- 
vantages. The first is that PIC codes suffer from high levels of statistical noise, 
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since they generally only deal with a relatively small number of particles (typi- 
cally < 10 6 ). Real physical systems do not exhibit anything like the same level of 
statistical noise, since they generally contain of order Avogadro's number (~ 1 24 ) 
of interacting particles. Another problem with PIC codes is that they do not han- 
dle charged particle collisions very well. The reason for this is that there are 
generally a large number of particles in each cell (for practical reasons), and 
the short range Coulomb fields of these particles tend to cancel one another out 
(recall that the electric field is only calculated at the cell vertices) . 
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9.1 Introduction 

Numerical methods which make use of random numbers are called Monte-Carlo 
methods — after the famous casino. The obvious applications of such methods are 
in stochastic physics: e.g., statistical thermodynamics. However, there are other, 
less obvious, applications: e.g., the evaluation of multi-dimensional integrals. 



9.2 Random Numbers 



No numerical algorithm can generate a truly random sequence of numbers, How- 
ever, there exist algorithms which generate repeating sequences of M (say) inte- 
gers which are, to a fairly good approximation, randomly distributed in the range 
to M — 1 . Here, M is a (hopefully) large integer. This type of sequence is termed 
psuedo-random. 

The most well-known algorithm for generating psuedo-random sequences of 
integers is the so-called linear congruental method. The formula linking the nth 
and (n + 1 )th integers in the sequence is 

I n+1 = (AI n + C)modM, (9.1) 

where A, C, and M are positive integer constants. The first number in the se- 
quence, the so-called "seed" value, is selected by the user. 

Consider an example case in which A = 7, C = 0, and M = 10. A typical 
sequence of numbers generated by formula (19. ID is 



I={3,1, 7,9,3,1, ■•■}. (9.2) 

Evidently, the above choice of values for A, C, and M is not a particularly good 
one, since the sequence repeats after only four iterations. However, if A, C, and 
M are properly chosen then the sequence is of maximal length (i.e., of length M), 
and approximately randomly distributed in the range to M — 1 . 
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The function listed below is an implementation of the linear congruental method. 

// random. cpp 

// Linear congruent ial psuedo-random number generator. 
// Generates psuedo-random sequence of integers in 
// range . . RANDMAX . 

#define RANDMAX 6074 // RANDMAX = M - 1 

int random (int seed = 0) 
{ 

static int next = 1; 
static int A = 106; 
static int C = 1283; 
static int M = 6075; 

if (seed) next = seed; 
next = next * A + C; 
return next % M; 

} 

The keyword static in front of a local variable declaration indicates that the 
program should preserve the value of that variable between function calls. In 
other words, if the static variable next has the value 999 on exit from function 
random then the next time this function is called next will have exactly the same 
value. Note that the values of non-static local variables are not preserved between 
function calls. The = in the first line of function random is a default value for the 
argument seed. In fact, random can be called in one of two ways. Firstly, random 
can be called with no argument: i.e., random () : in which case, seed is given the 
default value 0. Secondly, random can be called with an integer argument: i.e., 
random (n) : in which case, the value of seed is set to n. The first way of calling 
random just returns the next integer in the psuedo-random sequence. The second 
way seeds the sequence with the value n (i.e., h is set to n), and then returns the 
next integer in the sequence (i.e., I2). Note that the function prototype for random 
takes the form int random (int = 0): the = indicates that the argument is 
optional. 

The above function returns a pseudo-random integer in the range to RANDMAX 
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Figure 91: Plot of Xj versus x j+1 for j = 1, 10000. Here,, the Xj are random values, uniformly dis- 
tributed in the range to 1, generated using a linear congruental psuedo-random number generator 
characterized by A = 1 06, C = 1283, and M = 6075. 

(where RANDMAX takes the value M — 1 ) . In order to obtain a random variable x, 
uniformly distributed in the range to 1, we would write 

x = double (random ()) / double (RANDMAX); 

Now if x is truly random then there should be no correlation between successive 
values of x. Thus, a good way of testing our random number generator is to plot 
Xj versus Xj + i (where Xj corresponds to the jth number in the psuedo-random 
sequence) for many different values of j . For a good random number generator, 
the plotted points should densely fill the unit square. Moreover, there should be 
no discernible pattern in the distribution of points. 



Figure 1911 shows a correlation plot for the first 10000 x 5 —Xj + i pairs generated 
using a linear congruental psuedo-random number generator characterized by 



286 



9.2 Random Numbers 



9 MONTE-CARLO METHODS 




Figure 92: Plot of Xj versus x j+1 for j = 1, 10000. Here,, t/ie Xj are random values, uniformly dis- 
tributed in the range to 1, generated using a linear congruental psuedo-random number generator 
characterized by A = 1 07, C = 1283, and M = 6075. 



A = 106, C = 1283, and M = 6075. It can be seen that this is a poor choice of 
values for A, C, and M, since the pseudo-random sequence repeats after a few 
iterations, yielding Xj values which do not densely fill the interval to 1. 



Figure [92] shows a correlation plot for the first 10000 Xj-xj+i pairs generated 
using a linear congruental psuedo-random number generator characterized by 
A = 107, C = 1283, and M = 6075. It can be seen that this is a far better choice 
of values for A, C, and M, since the pseudo-random sequence is of maximal 
length, yielding Xj values which are fairly evenly distributed in the range to 1 . 
However, if we look carefully at Fig. [921 we can see that there is a slight tendency 
for the dots to line up in the horizontal and vertical directions. This indicates 
that the Xj are not quite randomly distributed: i.e., there is some correlation 
between successive Xj values. The problem is that M is too low: i.e., there is not 
a sufficiently wide selection of different Xj values in the interval to 1 . 
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Figure 93: Plot of Xj versus xj +1 for ) = 1, 10000. Here, t/ie Xj are random values, uniformly dis- 
tributed in the range to 1, generated using a linear congruental psuedo-random number generator 
characterized by A = 11 0351 5245, C = 12345, and M = 32768. 
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Figure [931 shows a correlation plot for the first 10000 Xj-Xj+i pairs generated 



using a linear congruental psuedo-random number generator characterized by 
A = 1103515245, C = 12345, and M = 32768. The clumping of points in this 
figure indicates that the Xj are again not quite randomly distributed. This time 
the problem is integer overflow: i.e., the values of A and M are sufficiently large 
that A I n > 10 32 — 1 for many integers in the pseudo-random sequence. Thus, the 
algorithm (19. ID is not being executed correctly. 

Integer overflow can be overcome using Schrange's algorithm. If y = ( A z) modM 
then 

f A(zmodq) -r(z/q) ify>0 
\ A (zmod q) - r (z/q) + M otherwise ' 

where q = M/A and r = M°/ A. The so-called Park and Miller method for gen- 
erating a pseudo-random sequence corresponds to a linear congruental method 
characterized by the values A = 16807, C = 0, and M = 2147483647. The func- 
tion listed below implements this method, using Schrange's algorithm to avoid 
integer overflow. 



// random. cpp 



// Park and Miller's psuedo-random number generator. 



#define RANDMAX 2147483646 // RANDMAX = M - 1 



int random (int seed = 0) 
{ 

static int next = 1; 
static int A = 16807; 

static int M = 2147483647; // 2~31 - 1 
static int q = 127773; //M/A 
static int r = 2836; // M 7, A 



if (seed) next = seed; 

next = A * (next °/ q) - r * (next / q) ; 
if (next < 0) next += M; 
return next ; 



Figure [941 shows a correlation plot for the first 10000 Xj-xj+i pairs generated 
using Park & Miller's method. We can now see no pattern whatsoever in the 
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Figure 94: PZot of Xj versus Xj + i /or j = 1 , 1 0000. Here, the Xj are random values, uniformly 
distributed in the range to 1, generated using Park & Miller's psuedo-random number generator. 
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plotted points. This indicates that the Xj are indeed randomly distributed in the 
range to 1 . From now on, we shall use Park & Miller's method to generate all the 
psuedo-random numbers needed in our investigation of Monte-Carlo methods. 



9.3 Distribution Functions 



Let P(x) dx represent the probability of finding the random variable x in the in- 
terval x to x + dx. Here, P(x) is termed a probability density. Note that P = 
corresponds to no chance, whereas P = 1 corresponds to certainty. Since it is 
certain that the value of x lies in the range — oo to +oo, probability densities are 
subject to the normalizing constraint 

P(x)dx = 1. (9.4) 

J— oo 



Suppose that we wish to construct a random variable x which is uniformly 
distributed in the range xi to xi. In other words, the probability density of x is 

pw=iy (x2 " x,) if ^*^. (95) 

{ otherwise 
Such a variable is constructed as follows 



x = xl + (x2 - xl) * double (random ()) / double (RANDMAX) ; 



There are two basic methods of constructing non-uniformly distributed ran- 
dom variables: i.e., the transformation method and the rejection method. We shall 
examine each of these methods in turn. 

Let us first consider the transformation method. Let y = f (x), where f is a 
known function, and x is a random variable. Suppose that the probability density 
of x is P x (x). What is the probability density, P y (y), of y? Our basic rule is the 
conservation of probability: 

|P x (x)dx| = |P y (y)dy|. (9.6) 
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In other words, the probability of finding x in the interval x to x + dx is the same 
as the probability of finding y in the interval y to y + dy . It follows that 

where f ' = df/dx. 

For example, consider the Poisson distribution: 

f ^ f e v if < -y < oo rn D , 

P ^ ) = {o otherwise ' (9 " 8) 

Let y = f (x) = — lnx, so that |f '| = 1 /x. Suppose that 

^ = { I SeWs'e 1 • 

It follows that 

? v {y) = ^ = x = e -\ (9.10) 

with x = corresponding to y = oo, and x = 1 corresponding to y = 0. We 
conclude that if 



x = double (random ()) / double (RANDMAX) ; 
y = - log (x) ; 



then y is distributed according to the Poisson distribution. 

The transformation method requires a differentiable probability distribution 
function. This is not always practical. In such cases, we can use the rejection 
method instead. 

Suppose that we desire a random variable y distributed with density P v [y ) in 
the range y m i n to y max . Let P ym ax be the maximum value of ?{y) in this range (see 
Fig. 1951). The rejection method is as follows. The variable y is sampled randomly 
in the range y m i n to y max - For each value of y we first evaluate P^(y). We next 
generate a random number x which is uniformly distributed in the range to 
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P 



max 



Figure 95: The rejection method. 

Pymax- Finally, if P y (y) < x then we reject the y value; otherwise, we keep it. If 
this prescription is followed then y will be distributed according to ? v [y ) . 

As an example, consider the Gaussian distribution: 

exp[(u -y) 2 /2(T 2 ] 



'In qj 

where y is the mean value of y, and a is the standard deviation. Let 

Umin = y -4a, 
Umax = V + 4 a, 



(9.11) 

(9.12) 
(9.13) 



since there is a negligible chance that y lies more than 4 standard deviations from 
its mean value. It follows that 

1 



y max 



2n<j } 



(9.14) 



with the maximum occurring at y = y . The function listed below employs the 
rejection method to return a random variable distributed according to a Gaussian 
distribution with mean mean and standard deviation sigma: 



// gaussian. cpp 

// Function to return random variable distributed 
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// according to Gaussian distribution with mean mean 
// and standard deviation sigma. 

#define RANDMAX 2147483646 

int random (int = 0) ; 

double gaussian (double mean, double sigma) 
{ 

double ymin = mean - 4. * sigma; 
double ymax = mean +4. * sigma; 
double Pymax = 1. / sqrt (2. * M_PI) / sigma; 

// Calculate random value uniformly distributed 
// in range ymin to ymax 

double y = ymin + (ymax - ymin) * double (random ()) / double (RANDMAX); 
// Calculate Py 

double Py = exp (- (y - mean) * (y - mean) / 2. / sigma / sigma) / 
sqrt (2. * M_PI) / sigma; 

// Calculate random value uniformly distributed in range to Pymax 
double x = Pymax * double (random ()) / double (RANDMAX); 

// If x > Py reject value and recalculate 
if (x > Py) return gaussian (mean, sigma) ; 
else return y; 

> 



Figure [96] illustrates the performance of the above function. It can be seen that 



the function successfully returns a random value distributed according to the 
Gaussian distribution. 



9.4 Monte-Carlo Integration 

Consider a one-dimensional integral: J** h f (x) dx. We can evaluate this integral 
numerically by dividing the interval xl to Xh into N identical subdivisions of width 

h=^p. (9.15) 
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Figure 96: A million values returned by function gaussian with mean = 5. and si gma = 1.25. The 
values are binned in 100 bins of width 0.1. The figure shows the number of points in each bin divided 
by a suitable normalization factor. A Gaussian curve is shown for comparison. 

Let Xi be the midpoint of the ith subdivision, and let f t = f (xj . Our approximation 
to the integral takes the form 



This integration method — which is known as the midpoint method — is not partic- 
ularly accurate, but is very easy to generalize to multi-dimensional integrals. 

What is the error associated with the midpoint method? Well, the error is 
the product of the error per subdivision, which is 0(h 2 ), and the number of 
subdivisions, which is 0(hr 1 ). The error per subdivision follows from the linear 
variation of f(x) within each subdivision. Thus, the overall error is 0(h 2 ) x 
0(H _1 ) = O(H). Since, h oc IN \ we can write 



Let us now consider a two-dimensional integral. For instance, the area en- 
closed by a curve. We can evaluate such an integral by dividing space into iden- 




(9.16) 




(9.17) 
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tical squares of dimension h, and then counting the number of squares, N (say), 
whose midpoints lie within the curve. Our approximation to the integral then 
takes the form 

A ~ IN h 2 . (9.18) 
This is the two-dimensional generalization of the midpoint method. 

What is the error associated with the midpoint method in two-dimensions? 
Well, the error is generated by those squares which are intersected by the curve. 
These squares either contribute wholly or not at all to the integral, depending on 
whether their midpoints lie within the curve. In reality, only those parts of the 
intersected squares which lie within the curve should contribute to the integral. 
Thus, the error is the product of the area of a given square, which is 0(H 2 ), and 
the number of squares intersected by the curve, which is 0(h ] ). Hence, the 
overall error is 0{h 2 ) x 0(hr ] ) = 0(h) = 0(1\T 1/2 ). It follows that we can write 

A = Nh 2 + 0(r 1/2 ). (9.19) 

Let us now consider a three-dimensional integral. For instance, the volume 
enclosed by a surface. We can evaluate such an integral by dividing space into 
identical cubes of dimension h, and then counting the number of cubes, N (say), 
whose midpoints lie within the surface. Our approximation to the integral then 
takes the form 

V ~ N h 3 . (9.20) 
This is the three-dimensional generalization of the midpoint method. 

What is the error associated with the midpoint method in three-dimensions? 
Well, the error is generated by those cubes which are intersected by the surface. 
These cubes either contribute wholly or not at all to the integral, depending on 
whether their midpoints lie within the surface. In reality, only those parts of the 
intersected cubes which lie within the surface should contribute to the integral. 
Thus, the error is the product of the volume of a given cube, which is 0(H 3 ), 
and the number of cubes intersected by the surface, which is 0(hr 2 ). Hence, the 
overall error is 0(h 3 ) x 0(h 2 ) = 0(h) = 0(N ] / 3 ). It follows that we can write 

V = NH 3 + 0(N- 1/3 ). (9.21) 
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Let us, finally, consider using the midpoint method to evaluate the volume, V, 
of a d-dimensional hypervolume enclosed by a (d — 1 ) -dimensional hypersurface. 
It is clear, from the above examples, that 

V = Nh d + 0(r 1/d ), (9.22) 

where N is the number of identical hypercubes into which the hypervolume is 
divided. Note the increasingly slow fall-off of the error with N as the dimension- 
ality, d, becomes greater. The explanation for this phenomenon is quite simple. 
Suppose that N = 10 6 . With N = 10 6 we can divide a unit line into (identi- 
cal) subdivisions whose linear extent is 1 6 , but we can only divide a unit area 
into subdivisions whose linear extent is 1 3 , and a unit volume into subdivisions 
whose linear extent is 10 2 . Thus, for a fixed number of subdivisions the grid 
spacing (and, hence, the integration error) increases dramatically with increas- 
ing dimension. 

Let us now consider the so-called Monte-Carlo method for evaluating multi- 
dimensional integrals. Consider, for example, the evaluation of the area, A, en- 
closed by a curve, C. Suppose that the curve lies wholly within some simple 
domain of area A', as illustrated in Fig. [97l Let us generate N' points which are 
randomly distributed throughout A'. Suppose that N of these points lie within 
curve C. Our estimate for the area enclosed by the curve is simply 

N 

A= 1 q 7 A / . (9.23) 

What is the error associated with the Monte-Carlo integration method? Well, 
each point has a probability p = A/A' of lying within the curve. Hence, the 
determination of whether a given point lies within the curve is like the measure- 
ment of a random variable x which has two possible values: 1 (corresponding to 
the point being inside the curve) with probability p, and (corresponding to the 
point being outside the curve) with probability 1 — p. If we make N' measure- 
ments of x (i.e., if we scatter N ' points throughout A') then the number of points 
lying within the curve is 

N = ^ x it (9.24) 

i=1 ,N' 
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Figure 97: The Monte-Carlo integration method. 
where Xt denotes the ith measurement of x. Now, the mean value of N is 



where 
Hence, 



i=1,N' 

x = 1 xp + Ox (1 — p) = p. 
N = N'p = N'^, 



(9.25) 



(9.26) 



(9.27) 



which is consistent with Eq. (19.231) . We conclude that, on average, a measure- 
ment of N leads to the correct answer. But, what is the scatter in such a measure- 
ment? Well, if cr represents the standard deviation of N then we have 

cr 2 = (N-N) 2 , 



which can also be written 



cr 2 = ^ (xt — x)(xj — x). 



(9.28) 



(9.29) 



However, (xt — x)(xj — x) equals (x — x) 2 if i = j, and equals zero, otherwise, 
since successive measurements of x are uncorrelated. Hence, 



Now, 



a 2 = N / (x-x) : 



(x — x) 2 = (x 2 — 2 x x + x 2 ) = x 2 — x , 



(9.30) 
(9.31) 
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and 

?=1 2 xp+0 2 x (1— p)=p. (9.32) 

Thus, 

(x-x) 2 = p-p 2 = p(1 -p), (9.33) 

giving 



<j = y /Wv{1 -p). (9.34) 
Finally, since the likely values of N lie in the range N = N ± cr, we can write 



N'4fl-4V (9.35) 



It follows from Eq. (19T23T) that 

A.A'^^p. (9.36) 

In other words, the error scales like (IN') ] / 2 . 

The Monte-Carlo method generalizes immediately to d-dimensions. For in- 
stance, consider a d-dimensional hypervolume V enclosed by a (d— 1 ) -dimensional 
hypersurface A. Suppose that A lies wholly within some simple hypervolume V. 
We can generate N' points randomly distributed throughout V. Let N be the 
number of these points which lie within A. It follows that our estimate for V is 
simply 

M 

V=— V. (9.37) 
Now, there is nothing in our derivation of Eq. (19.36D which depends on the fact 



that the integral in question is two-dimensional. Hence, we can generalize this 
equation to give 



V = V ,N ± VV(V/-V) 
N' v 7 ^ 7 

We conclude that the error associated with Monte-Carlo integration always scales 
like (N') _1 / 2 , irrespective of the dimensionality of the integral. 

We are now in a position to compare and contrast the midpoint and Monte- 
Carlo methods for evaluating multi-dimensional integrals. In the midpoint method, 
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Figure 98: Example calculation: volume of unit-radius 2-dimensional sphere enclosed in a close- 
fitting 2-dimensional cube. 

we fill space with an evenly spaced mesh of IN (say) points (i.e., the midpoints of 
the subdivisions), and the overall error scales like N 1/d , where d is the dimen- 
sionality of the integral. In the Monte-Carlo method, we fill space with N (say) 
randomly distributed points, and the overall error scales like N 1/2 , irrespective 
of the dimensionality of the integral. For a one-dimensional integral (d = 1), 
the midpoint method is more efficient than the Monte-Carlo method, since in the 
former case the error scales like N _1 , whereas in the latter the error scales like 
N 1/2 . For a two-dimensional integral (d = 2), the midpoint and Monte-Carlo 
methods are both equally efficient, since in both cases the error scales like N 1/2 . 
Finally, for a three-dimensional integral (d = 3), the midpoint method is less 
efficient than the Monte-Carlo method, since in the former case the error scales 
like N 1/3 , whereas in the latter the error scales like N 1/2 . We conclude that for 
a sufficiently high dimension integral the Monte-Carlo method is always going 
to be more efficient than an integration method (such as the midpoint method) 
which relies on a uniform grid. 

Up to now, we have only considered how the Monte-Carlo method can be 
employed to evaluate a rather special class of integrals in which the integrand 
function can only take the values or 1. However, the Monte-Carlo method can 
easily be adapted to evaluate more general integrals. Suppose that we wish to 
evaluate J f dV, where f is a general function and the domain of integration is of 
arbitrary dimension. We proceed by randomly scattering N points throughout the 
integration domain and calculating f at each point. Let x\ denote the ith point. 
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Figure 99: The integration error, e, versus the number of grid-points, N, for three integrals evaluated 
using the midpoint method. The integrals are the area of a unit-radius circle (solid curve), the volume 
of a unit-radius sphere (dotted curve), and the volume of a unit-radius 4-sphere (dashed curve). 

The Monte-Carlo approximation to the integral is simply 



fdv = — Y_ f(^) + o / 1 



N 



i=1,N 



(9.39) 



We end this section with an example calculation. Let us evaluate the volume 
of a unit-radius d-dimensional sphere, where d runs from 2 to 4, using both the 
midpoint and Monte-Carlo methods. For both methods, the domain of integration 
is a cube, centred on the sphere, which is such that the sphere just touches each 
face of the cube, as illustrated in Fig. [98]. 



Figure [99] shows the integration error associated with the midpoint method as 
a function of the number of grid-points, N . It can be seen that as the dimension- 
ality of the integral increases the error falls off much less rapidly as N increases. 



Figure [TOO] shows the integration error associated with the Monte-Carlo method 
as a function of the number of points, N . It can be seen that there is very little 
change in the rate at which the error falls off with increasing N as the dimen- 
sionality of the integral varies. Hence, as the dimensionality, d, increases the 



301 



9.5 The Ising Model 



9 MONTE-CARLO METHODS 







-2 

"2 -4 
o 

■ — 1 

-6 



2 4 6 8 10 

log 10 (N) 

Figure 100: The integration error, e, versus the number of points, N, for three integrals evaluated 
using the Monte-Carlo method. The integrals are the area of a unit-radius circle (solid curve), the 
volume of a unit-radius sphere (dotted curve), and the volume of a unit-radius 4-sphere (dashed 
curve). 

Monte- Carlo method eventually wins out over the midpoint method. 
9.5 The Ising Model 

Ferromagnetism arises when a collection of atomic spins align such that their asso- 
ciated magnetic moments all point in the same direction, yielding a net magnetic 
moment which is macroscopic in size. The simplest theoretical description of fer- 
romagnetism is called the Ising model. This model was invented by Wilhelm Lenz 
in 1920: it is named after Ernst Ising, a student of Lenz who chose the model as 
the subject of his doctoral dissertation in 1925. 

Consider N atoms in the presence of a z-directed magnetic field of strength 
H. Suppose that all atoms are identical spin-1 /2 systems. It follows that either 
Si = +1 (spin up) or Si = —1 (spin down), where Si is (twice) the z-component 
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of the ith atomic spin. The total energy of the system is written: 



E = -J Y_ s i s J ~ M-H Y_ Si - 



(9.40) 



<ij> i=1 ,N 



Here, < ij > refers to a sum over nearest neighbour pairs of atoms. Further- 
more, J is called the exchange energy, whereas u. is the atomic magnetic moment. 
Equation (19.401) is the essence of the Ising model. 

The physics of the Ising model is as follows. The first term on the right-hand 
side of Eq. (19.401) shows that the overall energy is lowered when neighbouring 
atomic spins are aligned. This effect is mostly due to the Pauli exclusion principle. 
Electrons cannot occupy the same quantum state, so two electrons on neighbour- 
ing atoms which have parallel spins (i.e., occupy the same orbital state) cannot 
come close together in space. No such restriction applies if the electrons have 
anti-parallel spins. Different spatial separations imply different electrostatic in- 
teraction energies, and the exchange energy, J, measures this difference. Note 
that since the exchange energy is electrostatic in origin, it can be quite large: i.e., 
J ~ 1 eV. This is far larger than the energy associated with the direct magnetic in- 
teraction between neighbouring atomic spins, which is only about 1 4 eV. How- 
ever, the exchange effect is very short-range; hence, the restriction to nearest 
neighbour interaction is quite realistic. 

Our first attempt to analyze the Ising model will employ a simplification known 
as the mean field approximation. The energy of the ith atom is written 



k=1,z 

where the sum is over the z nearest neighbours of atom i. The factor 1 /2 is 
needed to ensure that when we sum to obtain the total energy, 




(9.41) 




(9.42) 



i=1,N 



we do not count each pair of neighbouring atoms twice. 



We can write 



(J-HeffSi 



(9.43) 
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where 

H eff = H + -L Y s k . (9.44) 

Here, H e ff is the effective magnetic field, which is made up of two components: the 
external field, H, and the internal field generated by neighbouring atoms. 

Consider a single atom in a magnetic field H m . Suppose that the atom is in 
thermal equilibrium with a heat bath of temperature T. According to the well- 
known Boltzmann distribution, the mean spin of the atom is 

e +|3 |J.H m _ g-(3 |^H m 

5 = e +PnH m + e -|3 H H m ' (9.45) 

where (3 = 1 /kT, and k is the Boltzmann constant. The above expression follows 
because the energy of the "spin up" state (s = +1) is — U-H m , whereas the energy 
of the "spin down" state (s = — 1) is +u.H m . Hence, 

s = tanh((3(iH m ). (9.46) 

Let us assume that all atoms have identical spins: i.e., s\ = s. This assumption 
is known as the "mean field approximation". We can write 

Heff = H + (9.47) 



Finally, we can combine Eqs. (I9.46D and (19.471) (identifying H m and H e ff) to ob- 
tain 

s = tanh{(3 (iH + (3 z J s/2} . (9.48) 

Note that the heat bath in which a given atom is immersed is simply the rest of 
the atoms. Hence, T is the temperature of the atomic array. It is helpful to define 
the critical temperature, 

T c = fl (9.49) 



and the critical magnetic field, 



H c = ^ = *l. (9.50) 
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Equation (19.481) reduces to 

§ = tanh {T(S + s ")}' (9 - 51) 

The above equation cannot be solved analytically. However, it is fairly easily to 
solve numerically using the following iteration scheme: 

s i+ i = tanh [ y (ff + s_i ) } • (9 - 52) 
The above formula is iterated until sVi — > s\. 
It is helpful to define the net magnetization, 

M = \i Y_ s i = M-N s, (9.53) 

i=1,N 

the net energy, 

E=X^ = " N kT c + s) s, (9.54) 



i=1,N 

and the heat capacity, 



c 



C = ^. (9.55) 
d I 



Figures I102|. llOlj. and I103I show the net magnetization, net energy, and heat 



capacity calculated from the iteration formula (19.52D in the absence of an exter 



nal magnetic field (i.e., with H = 0). It can be seen that below the critical (or 
"Curie") temperature, T c , there is spontaneous magnetization: i.e., the exchange 
effect is sufficiently large to cause neighbouring atomic spins to spontaneously 
align. On the other hand, thermal fluctuations completely eliminate any align- 
ment above the critical temperature. Moreover, at the critical temperature there 
is a discontinuity in the first derivative of the energy, E, with respect to the tem- 
perature, T. This discontinuity generates a downward jump in the heat capacity, 
C, at T = T c . The sudden loss of spontaneous magnetization as the temperature 
exceeds the critical temperature is a type of phase transition. 

Now, according to the conventional classification of phase transitions, a transi- 
tion is first-order if the energy is discontinuous with respect to the order parame- 
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1 ^ - 1 1 

0.5 1 1.5 2 

T / T c 

Figure 101: The net magnetization, M, of a collection of N ferromagnetic atoms as a function of the 
temperature, T, in the absence of an external magnetic field. Calculation performed using the mean 
field approximation. 




0.5 1 1.5 2 



T / T c 

Figure 102: The net energy, E, of a collection of ~N ferromagnetic atoms as a function of the temper- 
ature, T, in the absence of an external magnetic field. Calculation performed using the mean field 
approximation. 
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Figure 103: The heat capacity, C, of a collection of N ferromagnetic atoms as a function of the 
temperature, T, in the absence of an external magnetic field. Calculation performed using the mean 
field approximation. 

ter (i.e., in this case, the temperature), and second-order if the energy is continu- 
ous, but its first derivative with respect to the order parameter is discontinuous, 
etc. We conclude that the loss of spontaneous magnetization in a ferromagnetic 
material as the temperature exceeds the critical temperature is a second-order 
phase transition. 

In order to see an example of a first-order phase transition, let us examine the 
behaviour of the magnetization, M, as the external field, H, is varied at constant 
temperature, T. 

Figures 11041 and 11051 show the magnetization, M, and energy, E, versus ex- 
ternal field-strength, H, calculated from the iteration formula (19.521) at some 



constant temperature, T, which is less than the critical temperature, T c . It can be 
seen that E is discontinuous, indicating the presence of a first-order phase transi- 
tion. Moreover, the system exhibits hysteresis — meta-stable states exist within a 
certain range of H values, and the magnetization of the system at fixed T and H 
(within the aforementioned range) depends on its past history: i.e., on whether 
H was increasing or decreasing when it entered the meta-stable range. 
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Figure 104: The net magnetization, M, of a collection of N ferromagnetic atoms as a function of the 
external magnetic field, H, at constant temperature, T < T c . Calculation performed using the mean 
field approximation. 



meta-stable states 




-2-10 1 2 



H / H c 

Figure 105: The net energy, ~E, of a collection of N ferromagnetic atoms as a function of the external 
magnetic field, H, at constant temperature, T < T c . Calculation performed using the mean field 
approximation. 
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H / H c 

Figure 106: The net magnetization, M, of a collection of N ferromagnetic atoms as a function of the 
external magnetic field, H, at constant temperature, T = T c . Calculation performed using the mean 
field approximation. 




H / H c 

Figure 107: The net energy, E, of a collection of N ferromagnetic atoms as a function of the external 
magnetic field, H, at constant temperature, T = T c . Calculation performed using the mean field 
approximation. 
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L 



L 

Figure 108: A two-dimensional array of atoms. 
Figures 11061 and 11071 show the magnetization, M, and energy, E, versus exter- 



nal field-strength, H, calculated from the iteration formula (I9.52D at a constant 
temperature, T, which is equal to the critical temperature, T c . It can be seen that 
E is now continuous, and there are no meta-stable states. We conclude that first- 
order phase transitions and hysteresis only occur, as the external field-strength is 
varied, when the temperature lies below the critical temperature: i.e., when the 
ferromagnetic material in question is capable of spontaneous magnetization. 

The above calculations, which are based on the mean field approximation, 
correctly predict the existence of first- and second-order phase transitions when 
H^O and H = 0, respectively. However, these calculations get some of the details 
of the second-order phase transition wrong. In order to do a better job, we must 
abandon the mean field approximation and adopt a Monte-Carlo approach. 

Let us consider a two-dimensional square array of atoms. Let L be the size of 
the array, and N = L 2 the number of atoms in the array, as shown in Fig. 11081 
The Monte-Carlo approach to the Ising model, which completely avoids the use 
of the mean field approximation, is based on the following algorithm: 

• Step through each atom in the array in turn: 

- For a given atom, evaluate the change in energy of the system, AE, when 
the atomic spin is flipped. 

- If AE < then flip the spin. 

- If AE > then flip the spin with probability P = exp(— (3 AE). 
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• Repeat the process many times until thermal equilibrium is achieved. 

The purpose of the algorithm is to shuffle through all possible states of the sys- 
tem, and to ensure that the system occupies a given state with the Boltzmann 
probability: i.e., with a probability proportional to exp(— |3E), where E is the 
energy of the state. 

In order to demonstrate that the above algorithm is correct, let us consider 
flipping the spin of the ith atom. Suppose that this operation causes the system 
to make a transition from state a (energy, E a ) to state b (energy, E b ). Suppose, 
further, that E Q < E b . According to the above algorithm, the probability of a 
transition from state a to state b is 

P a ^ b = exp[-|3(E b -E Q )], (9.56) 

whereas the probability of a transition from state b to state a is 

Pb->a = 1- (9-57) 
In thermal equilibrium, the well-known principal of detailed balance implies that 

Pa Pa— >b = Pb Pb— >a> 

(9.58) 

where P a is the probability that the system occupies state a, and P b is the prob- 
ability that the system occupies state b. Equation (I9.58D simply states that in 



thermal equilibrium the rate at which the system makes transitions from state a 
to state b is equal to the rate at which the system makes reverse transitions. The 
previous equation can be rearranged to give 

^ = exp[-(3(E b -E Q )], (9.59) 

' a 

which is consistent with the Boltzmann distribution. 

Now, each atom in our array has four nearest neighbours, except for atoms on 
the edge of the array, which have less than four neighbours. We can eliminate 
this annoying special behaviour by adopting periodic boundary conditions: i.e., 
by identifying opposite edges of the array. Indeed, we can think of the array as 
existing on the surface of a torus. 



311 



9.5 The Ising Model 



9 MONTE-CARLO METHODS 




T / T 

Figure 109: The net magnetization, Wl, of a 5 x 5 array of ferromagnetic atoms as a function of the 
temperature, T, in the absence of an external magnetic field. Monte-Carlo simulation. 



It is helpful to define 



Now, according to mean field theory, 



J 

k* 



The evaluation of 



2k 
C = lim 



2 To. 



AE 



(9.60) 



(9.61) 



(9.62) 

AT^O AT 

via the direct method is difficult due to statistical noise in the energy, E. Instead, 
we can make use of a standard result in equilibrium statistical thermodynamics: 



C 



kT 2 ' 



(9.63) 



where <te is the standard deviation of fluctuations in E. Fortunately, it is fairly 
easy to evaluate u^: we can simply employ the standard deviation in E from step 
to step in our Monte-Carlo iteration scheme. 



Figures 11091 - 41 161 show magnetization and heat capacity versus temperature 
curves for E = 5, 10, 20, and 40 in the absence of an external magnetic field. In 
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Figure 110: The heat capacity, C, of a 5 x 5 array of ferromagnetic atoms as a function of the 
temperature, T, in the absence of an external magnetic field. Monte-Carlo simulation. The solid 
curve shows the heat capacity calculated from Eq. <\9. 62\) , whereas the dotted curve shows the heat 
capacity calculated from Eq. A9.63P . 

all cases, the Monte-Carlo simulation is iterated 5000 times, and the first 1000 
iterations are discarded when evaluating <te (in order to allow the system to 
attain thermal equilibrium) . The two-dimensional array of atoms is initialized in 
a fully aligned state for each different value of the temperature. Since there is no 
external magnetic field, it is irrelevant whether the magnetization, M, is positive 
or negative. Hence, M is replaced by |M| in all plots. 

Note that the M versus T curves generated by the Monte-Carlo simulations 
look very much like those predicted by the mean field model. The resemblance 
increases as the size, L, of the atomic array increases. The major difference is the 
presence of a magnetization "tail" for T > T c in the Monte-Carlo simulations: i.e., 
in the Monte-Carlo simulations the spontaneous magnetization does not collapse 
to zero once the critical temperature is exceeded — there is a small lingering mag- 
netization for T > T c . The C versus T curves show the heat capacity calculated 
directly (i.e., C = AE/AT), and via the identity C = a^/kT 2 . The latter method 
of calculation is clearly far superior, since it generates significantly less statistical 
noise. Note that the heat capacity peaks at the critical temperature: i.e., unlike 
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Figure 111: The net magnetization, M, of a 10 x 10 array of ferromagnetic atoms as a function of 
the temperature, T, in the absence of an external magnetic field. Monte-Carlo simulation. 

the mean field model, C is not zero for T > T c . This effect is due to the residual 
magnetization present when T > T c . 

Our best estimate for T c is obtained from the location of the peak in the C ver- 
sus T curve in Fig. 11161 We obtain T c = 2.27 Tq. Recall that the mean field model 



yields T c = 2 To. The exact answer for a two-dimensional array of ferromagnetic 
atoms is 

j <=wvtvT) =227Jo ' (9 - 64) 

which is consistent with our Monte-Carlo calculations. The above analytic result 



was first obtained by Onsager in 1944a Incidentally Onsager's analytic solution 
of the 2-D Ising model is one of the most complicated and involved calculations 
in all of theoretical physics. Needless to say no one has ever been able to find an 
analytic solution of the Ising model in more than two dimensions. 



Note, from Figs. [HOI I112j 11141 and EH that the height of the peak in the heat 



capacity curve at T = T c increases with increasing array size, L. Indeed, a close 
examination of these figures yields C max /N k = 0.95 for L = 5, C max /N k = 1 .34 



39 



L. Onsager, Phys. Rev. 65, 117 (1944). 
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O 0.5 




Figure 112: The heat capacity, C, of a 10 x 10 array of ferromagnetic atoms as a function of the 
temperature, T, in the absence of an external magnetic field. Monte-Carlo simulation. The solid curve 
shows the heat capacity calculated from Eq. (\9.62\) , whereas the dotted curve shows the heat capacity 
calculated from Eq. (T9.63P . 

for L = 10, C m ax/Nk = 1.77 for L = 20, and C max /Nk = 2.16 for L = 40. 
Figure 11171 shows C max /M k plotted against In L for L = 5, 10, 20, and 40. It 
can be seen that the points lie on a very convincing straight-line, which strongly 
suggests that 

Cmax 

w 



oc InL. 



(9.65) 



Of course, for physical systems, L ~ \/Na ~ 10 12 , where N A is Avogadro's num- 
ber. Hence, C is effectively singular at the critical temperature (since lnN A 1), 
as sketched in Fig. 11181 This observation leads us to revise our definition of a 
second-order phase transition. It turns out that actual discontinuities in the heat 
capacity almost never occur. Instead, second-order phase transitions are charac- 
terized by a local quasi-singularity in the heat capacity. 



Recall, from Eq. (19.63D , that the typical amplitude of energy fluctuations is 
proportional to the square-root of the heat capacity (i.e., cr^ oc vC). It follows 
that the amplitude of energy fluctuations becomes extremely large in the vicinity 
of a second-order phase transition. 
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Figure 113: The net magnetization, M, of a 20 x 20 array of ferromagnetic atoms as a function of 
the temperature, T, in the absence of an external magnetic field. Monte-Carlo simulation. 




Figure 114: The heat capacity, C, of a 20 x 20 array of ferromagnetic atoms as a function of the 
temperature, T, in the absence of an external magnetic field. Monte-Carlo simulation. The solid curve 
shows the heat capacity calculated from Eq. (\9.62\) , whereas the dotted curve shows the heat capacity 
calculated from Eq. ([9.63P . 
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Figure 115: The net magnetization, M, of a 40 x 40 array of ferromagnetic atoms as a function of 
the temperature, T, in the absence of an external magnetic field. Monte-Carlo simulation. 



Now, the main difference between our mean field and Monte-Carlo calcula- 
tions is the existence of residual magnetization for T > T c in the latter case. 
Figures 111 91 - 11231 show the magnetization pattern of a 40 x 40 array of ferromag- 



netic atoms, in thermal equilibrium and in the absence of an external magnetic 
field, calculated at various temperatures. It can be seen that for T = 20 To the 
pattern is essentially random. However, for T = 5 To, small clumps appear in the 
pattern. For T = 3 To, the clumps are somewhat bigger. For T = 2.32 To, which 
is just above the critical temperature, the clumps are global in extent. Finally, 
for T = 1.8 To, which is a little below the critical temperature, there is almost 
complete alignment of the atomic spins. 

The problem with the mean field model is that it assumes that all atoms are sit- 
uated in identical environments. Hence, if the exchange effect is not sufficiently 
large to cause global alignment of the atomic spins then there is no alignment 
at all. What actually happens when the temperature exceeds the critical tem- 
perature is that global alignment disappears, but local alignment (i.e., clumping) 
remains. Clumps are only eliminated by thermal fluctuations once the temper- 
ature is significantly greater than the critical temperature. Atoms in the middle 
of the clumps are situated in a different environment than atoms on the clump 
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Figure 116: The heat capacity, C, of a 40 x 40 array of ferromagnetic atoms as a function of the 
temperature, T, in the absence of an external magnetic field. Monte-Carlo simulation. The solid curve 
shows the heat capacity calculated from Eq. (\9.62\) , whereas the dotted curve shows the heat capacity 
calculated from Eq. (T9.63P . 



boundaries. Hence, clumps cannot occur in the mean field model. 
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Figure 117: The peak value of the heat capacity (normalized by N k) versus the logarithm of the 
array size for a two-dimensional array of ferromagnetic atoms in the absence of an external magnetic 
field. Monte-Carlo simulation. 




Figure 118: A sketch of the expected variation of the heat capacity versus the temperature for a 
physical two-dimensional ferromagnetic system. 
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Figure 119: Magnetization pattern of a 40 x 40 array of ferromagnetic atoms in thermal equilib- 
rium and in the absence of an external magnetic field. Monte-Carlo calculation with T = 20 T . 
Black/white squares indicate atoms magnetized in plus/minus z-direction, respectively. 




Figure 120: Magnetization pattern of a 40 x 40 array of ferromagnetic atoms in thermal equilibrium 
and in the absence of an external magnetic field. Monte-Carlo calculation with T = 5 T . Black/white 
squares indicate atoms magnetized in plus/minus z-direction, respectively. 
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Figure 121: Magnetization pattern of a 40 x 40 array of ferromagnetic atoms in thermal equilibrium 
and in the absence of an external magnetic field. Monte-Carlo calculation with T = 3 T . Black/white 
squares indicate atoms magnetized in plus/minus z-direction, respectively. 
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Figure 122: Magnetization pattern of a 40 x 40 array of ferromagnetic atoms in thermal equilib- 
rium and in the absence of an external magnetic field. Monte-Carlo calculation with T = 2.32 T . 
Black/white squares indicate atoms magnetized in plus/minus z-direction, respectively. 
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Figure 123: Magnetization pattern of a 40 x 40 array of ferromagnetic atoms in thermal equilib- 
rium and in the absence of an external magnetic field. Monte-Carlo calculation with T = 1 .8 T . 
Black/white squares indicate atoms magnetized in plus/minus z-direction, respectively. 
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